Interesting read on the way LLM bots retrieve pages from a website
Explanations are clear precise and surgical
https://surfacedby.com/blog/nginx-logs-ai-traffic-vs-referral-traffic
#LLM #AI #slop #nginx #traffic #programming #referral #traffic #networking #robots #txt #claude #chatgpt #bing #meta #metaAI









