SophieMy website is for humans, and I don't want Google's AI ramen
Charles FevalHey @sophie, can I suggest to update your robots.txt? Right now it tells any scraper that it's fine to roam you cute little garden.
@Cfe84 The reason I haven't is because I didn't think that was very effective at actually keeping the LLM scrapers out - they seem to just do what they want
@sophie that's fair, open AI and anthropic seem to not care at all, Google seem to respect it. I think it's more of a matter of displaying the "no trespassing" sign, people might still ignore it but at least they can't claim ignorance.
(with that being said I agree with 100% of your article).