Apr 20, 2025
glad we're at the stage of our cyberpunk hell-timeline that we have corporate botnets DDoSing the free software communities that they rely on, leading to an arms race between the biggest companies in the world and a virtual anime person developing a proof-of-work proxy with an anime girl mascot that's now deployed by the united nations
Botnet Part 2: The Web is Broken

I guess you have all heard about the growing problem of AI companies trying to aggressively collect whatever data they can get their hands on to train their models. This has caused an explosive surge in web crawlers relentlessly hitting servers big and small. But who runs these crawlers? Turns out — it could be you!

Jan Wildeboer's Blog
Show threadjra
@mia I both love and hate this. Meanwhile, the urge to purge most interactions with the modern web and simply rely on lynx & co. is increasing with each passing week.
Apr 24, 2025 at 8:48amWeb
Show threadJessie Nabein Apr 27, 2025

@jra @mia this is one of my personal dilemmas too... We exclude old web and text only browsers, both of which have legitimate uses in our modern world.

So far i've found it easy to do ASN and user agent blocks instead.

Show threadjraApr 27, 2025

@jessienab @mia My main issue with both user-agent blocks and a monster truck solution like Anubis is the fact that we're just sticking acid band-aids on top of acid band-aids while the older ones rot and decay. They might do the job for a while, but there's a cost. And the band-aid supply must be constant.

GenAI bots are a remarkably stupid example of scorched earth development, but, sadly, there's currently no way to fight them (in a technical sense) without searing the web a bit more ourselves.