TeaCloudflare announces AI Labyrinth, which uses AI-generated content to confuse and waste the resources of AI Crawlers and bots that ignore “no crawl” directives.
lily33while allowing legitimate users and verified crawlers to browse normally.
What is a “verified crawler” though? What I worry about is, is it only big companies like Google that are allowed to have them now?
I assume a crawler which adheres to robots.txt
I would love to think so. But the word “verified” suggests more.
killeronthecornerIP verification is a not uncommon method for commercial crawlers
I dunno. I don’t find any sympathy with any of these fuckers though. this is not a generally useful technology, it is not something the average person ever needs to see, and honestly, just fuck em. Fuck anyone messing with open source to engorge the garbage dispenser.
Any accessibility service will also see the “hidden links”, and while a blind person with a screen reader will notice if they wonder off into generated pages, it will waste their time too.
Also, I don’t know about you, but I absolutely have a use for crawling X, Google maps, Reddit, YouTube, and getting information from there without interacting with the service myself.
yeah. it’s pretty fucked. hopefully it’s temporary.
so do we make everything inaccessible to everyone, or just inaccessible to disabled people? we don’t have a way to include them yet. we should work on it, but we are not the ones who fucked accessibility.
yeah. search engine web crawlers are a public service. they are responsible. but we are in a conflict. we must struggle tooth and nail against capital for every nice thing.
I’d assume they’re using aria tags to hide the links from screen readers, at least that’s what the article seems to imply.
Cloudflare isn’t the best at blocking things. As long as your crawler isn’t horribly misconfigured you shouldn’t have much issues.