#openai | Yaksh Bariya

I'm not sure how long this "AI bubble" is going to last, but as a webmaster I'm now very tired with OpenAI, ByteDance, and other AI companies absolutely wrecking a chaos on public internet. Their crawlers don't give a shit about how many requests they do in a second. Yesterday it has come into my account that #openai's crawlers have moved from using "OpenAI" string in their user agent to "Intel MacOS" useragent to bypass people trying to protect their sites from rampant crawling. The server running MediaWiki for wiki.termux.dev was hit from 27k+ ip addresses in the past 12 hour window coming all from different IP ranges. Also what's surprising is that ByteDance IP ranges changed the user agent string the same time as OpenAI's crawlers. This can't be a coincidence, but seems like the entire AI industry is run by bunch of people who don't want to be responsible for their actions and want to make money off of blatant scraping and forcing a lot of small community sites and forms to shutdown. It is surely not a coincidence that a we started receiving bot requests from both ByteDance's crawlers and OpenAI's with the same user agent? Also a good chunk of these requests are coming from residental IPs. It is very likely that a bunch of ISPs are either giving their residental IPs to these AI companies, or they have resorted to buying residental connections for large scale scraping. Generally DDoS from residental IP ranges means that the attack is carried down by some script kiddie or some wannabe "cool hacker" who is operating a botnet and wants to demonstrate the scale of their attack to potential clients. There are also a lot of requests which seem to be coming from no-name ASNs which have no information available publicaly about what they do. Which is too fishy for such a large number of requests. This is not the full report, I'm planning to do a full report about this soon. Just wanted to rant out my frustration with me spending around 2-3 hours yesterday when I should be sleeping just because some people don't believe in the collective good use of internet bandwidth. I'm pretty sure there are a lot of other people as well who are dealing with this nonsense of AI crawlers