ClarkesworldIf you've been blocking AI from scraping your website, there's another one to add. This time it's Google.
I've updated my post on the subject.
https://neil-clarke.com/block-the-bots-that-feed-ai-models-by-scraping-your-website/
John Morahan@clarkesworld Note that according to the linked Google article, "Google-Extended" isn't an actual crawler with its own user agent string - rather, it's a name that Google's existing crawlers interpret as a request to opt out of AI when they see it in robots.txt. So blocking it at the firewall etc would have no effect.
@jmorahan Hmmm. Ok. Updating document.