Matthias OttFeb 1
Alright my web friends! 👋 Hands up who has experienced a surge in (LLM) bot traffic recently and maybe even had to take steps against them? I’m writing a blog post about this atm and it would be great to hear whether others are experiencing the same with their #blogs and personal #websites. #RT == 💚
Show threadWenzel Massag

@matthiasott I had a short but interesting conversation with the owner of a hosting company.

Here is the gist of it:

Part of the explanation for the surge in traffic can be an endpoint that takes variables via HTTP-GET, because the bots then try all possible combinations of variables. By making content only available via one URL per piece, say, articles cannot be linked to with a …?related=tag1,tag2,tag3… you should be able to reduce the load.

Feb 2 at 9:04amWeb
Show threadMatthias OttFeb 2
@stairjoke That’s really interesting – because this is actually part of what I did to reduce the load a bit! My notes page used to work with multiple tags as URL params. And I indeed saw a lot of requests by bots trying all kinds of combinations. I now reduced this to one tag, which already helped a bit. Although I also did a lot of other stuff, so I can’t say for sure how much which step helped exactly. 😅