I found that the fail2ban setup I had been using to ban scrapers didn't actually ban scrapers. I'm not sure if it never worked before or if there was a change that came with the new Debian stable. The key is that the iptables tools in Debian are a compatibility layer that call nft in the background. And the nft firewall tables that fail2ban created contained sets that didn't have the "interval" flag set, which means that fail2ban couldn't ban IP address ranges (anymore?). I fixed that setup, yesterday, and it seems to work. π
The reason I hadn't realised that it didn't work was that fail2ban still listed all the entries as banned but the firewall didn't actually ban them. π
This is where I found out I had a problem:
https://alexschroeder.ch/view/2025-12-23-santa-bots
This is the updated description of my setup:
https://transjovian.org/view/fight-bots/index
It seems that Drew DeVault's article kicked some journalists into awareness of the problem that's been haunting me for years. It's just become more urgent over time. FOSS infrastructure is under attack by AI companies mentions KDE, Gnome, LWN, Fedora, Inkscape, Diaspora, Read the Docs.
My first encounter with badly written bots was back in 2018. At the time I though it was just inept programmers. Now I know that it's callous, dead-inside programmers.
Alex Schroeder
Hacker News
