This is pretty wild — our friends at "Read The Docs" saw file download traffic drop by 75% after blocking AI bots.

https://about.readthedocs.com/blog/2024/07/ai-crawlers-abuse/

AI crawlers need to be more respectful

We talk a bit about the AI crawler abuse we are seeing at Read the Docs, and warn that this behavior is not sustainable.

Read the Docs
@stefan ugh.... so annoying that these AI companies are just chowing down on everyone's public data... which I personally don't mind to an extent but when they then sit down and turn around and sell it to everyone else the stuff they ingested for free... after actively knowing how much you are scraping, they really need to at least donate back to the sources they are scraping AT MINIMUM