trash mouseOct 7, 2025

Almost 29 million requests from AI crawlers defeated by essentially one simple check: if the user agent contains Chrome/ or Firefox/, and doesn't have sec-fetch-mode, it's going into the maze.

Billions of dollars poured into AI, yet, their crawlers are broken by two ifs in an nginx config.

If this all wasn't so sad, I'd laugh.

Post by iocaine powder, @[email protected]

#iocaine has been up for 4days 15h 5m 28s, and spent 1day 6h 26m 31s dealing with - *gestures hands wildly* - [everything](https://monitor.madhouse-project.o…

come-from.mad-scientist.club
Show thread4censord Oct 7, 2025
@algernon do you have these nginx ifs available somewhere?
because while i have planned to deploy iocaine at some point, i havent had the time to do it yet, but i could just plop something into nginx configs for now
Show threadtrash mouse

@4censord Not yet. It's in a WIP blog post I planned to publish last weekend. I hope to have it up in a few days.

It goes something like this:

set $sfm "default" if ($http_user_agent ~ "(Chrome/|Firefox/)") { set $sfm $http_sec_fetch_mode; } if ($sfm = "") { return 418; }

(Note: I cobbled this together a couple of days ago, but haven't had the chance to test it yet.)

Oct 7, 2025 at 10:57pmWeb
Show thread4censord Oct 7, 2025
@algernon oh i'm looking forward to that post then!
in the meantime i'll try these ifs and see what happens