Defeatism is a form of surrender. Cynicism is surrender. Despair is surrender. Nihilism is surrender.

Our job is to •care• and to •keep caring• and to •keep doing and keep building• and to •endure• longer than them.

@inthehands One of the things I've done recently is to bring enforcing robots.txt within my webserver engine. The /robots.txt itself still exists; the vast majority of it is a list of bots that are `Disallow: /` .

I still get a few of these bots attempting to hit the site, so it's definitely doing something.

@inthehands All this, and a little more! When Google *does* start ignoring robots.txt and other mechanisms, that's another victory for us, not them, even if it means we have to react to it.

Not all of Google's infrastructure is servers in a giant building, or software systems running on top of it, or even offices full of stressed out tech workers. Part of their infrastructure, the cladding on the castle walls, is their false pretense of being good citizens on the internet. When we call their bluff and they eventually drop the pretense, that's us getting them to tear down the outer layers of the castle themselves. We know what they are, and we can make them admit it, and that's power.

@inthehands I know of at least one professional artist who has deliberately poisoned their images, in an attempt to deter AI scraping (mostly because the scrapers blast her small site and effectively DoS it). If they follow robots.txt, they're not affected... but they were already ignoring robots.txt

I just read an IARPA paper that said poisoning as little as .1% of training data can disrupt a model. If content creators choose to deliberately poison content that they ask not to be scraped, it might be a nice way to deter bad behavior.

The tools I know of work on imagery, but with effort people may come up with stuff that works on data as well. E.g., burying base64-encoded malicious prompts in your text, posting tables as poisoned images rather than text, etc.

Seems like we should start organizing and taking firm action now, before AI companies start buying politicians and making such defenses illegal.

@inthehands Google executives don't care about consent. Google executives care about their bottom line.

Copyright class action seems most promising. Along with adversarial material to poison their scrapers

@inthehands In that case, it's better if we send them poisoned data instead, using iocaine [0] or nepentheses [1].

[0]: https://iocaine.madhouse-project.org
[1]: https://zadzmo.org/code/nepenthes/

@inthehands

I’m mostly worried about their “agentic” part, because that sounds like new infrastructure with possibly different user agents etc., so harder to ban, and I’m 💯 sure it will DEFINITELY have no “social contract” whatsoever.

@inthehands You actually can kinda enforce noindex / robots.txt!

If you set a page as no index, consider adding somewhere in it a random, non-displayed, non-screenreadable link titled something human-obvious like “ban-me-382972.html”. Add a human readable warning in the HTML comments. Then a little quick HAProxy config can IP-ban anyone who nonetheless tries to load it. It worked like magic on my server to reduce AI bot load. And all they had to do to avoid a ban was respect my robots.txt!

@ShadowJonathan @inthehands agree. I don't think defense is the best reaction to sustain a healthy internet. this rhetoric has been untrue since... Google (other similar corps).

random offensive approach such as collective data poisoning, public exposé, factual based journalism, education, jailtime, guillotine & other accountability and positive encouragement should coexist to foster the internet to recover better

@ShadowJonathan @inthehands they alsof are pretty involved with the contents of the standard (75% of the authors), so luring the crawler into a pit of crappy data is probably your only way to protest besides avoiding to hand any of your money and attention to them.

Link: https://datatracker.ietf.org/doc/html/rfc9309