MistyMar 25, 2025

I really can't overemphasize how destructive this is to the open web. The web is built just as much on a consensus on how to fairly do things as it is on technical measures keep things going, and AI scrapers have blown that up. https://go-to-hellman.blogspot.com/2025/03/ai-bots-are-destroying-open-access.html

I know Molly White wrote a passionate post recently asking people not to throw away open access just to stop AI scrapers, but many people *don't have a choice* anymore. If the old deal is broken, some people may be forced to take things private.

AI bots are destroying Open Access

There's a war going on on the Internet. AI companies with billions to burn are hard at work destroying the websites of libraries, archives, ...

Show threadGenders: ♾️, 🟪⬛🟩; Soni L.Mar 25, 2025
@misty do we have... any viable p2p-based solutions to this problem? maybe?
Show threadBilly Smith

@SoniEx2 @misty

What about only providing torrent/magnet links to the full publications?

Provide an abstract, yes, but if you want the full papers you have to use torrenting. :D

While this would change the way that things are accessed, it would also directly increase the bandwidth bills of the LLM's. :D

Do it enough that they become more unprofitable to train. :D

Mar 26, 2025 at 12:12amWeb
Show threadGenders: ♾️, 🟪⬛🟩; Soni L.Mar 26, 2025
@BillySmith @misty that would be a good start. but then there's git, etc, and we don't think we have any viable options for those yet?