404 MediaJan 23

A web developer has created and released an infinitely generating "tar pit" designed to trap AI training bots in an endless maze: "Just me unleashing shear unadulterated rage at how things are going"

https://www.404media.co/developer-creates-infinite-maze-to-trap-ai-crawlers-in/

Developer Creates Infinite Maze That Traps AI Training Bots

"Nepenthes generates random links that always point back to itself - the crawler downloads those new links. Nepenthes happily just returns more and more lists of links pointing back to itself."

404 Media
Show threadJesse
@404mediaco how can this work? Even in my computer science classes 15 years ago this was already mentioned and explained how to counter. Usually by randomly jumping back iirc
Jan 23 at 3:50pmIvory for iOS
Show threadkhmJan 23
malicious scrapers put a hilariously low amount of effort into their tools, because quality costs money and if they spend any their margins take a hit

CC: @404mediaco@mastodon.social