Patrick SamphireSep 29, 2023

To stop AIs scraping your website for content, add this to your robots.txt file on your website.

Thanks to Neil Clarke for most of these.

User-agent: CCBot
Disallow: /

User-agent: ChatGPT-User
Disallow: /

User-agent: GPTBot
Disallow: /

User-agent: Google-Extended
Disallow: /

User-agent: Omgilibot
Disallow: /

User-Agent: FacebookBot
Disallow: /

Show threadEric Vey

@patricksamphire

This is like submitting your telephone number to a do not call list: it only stops the honest ones.

But since scraping copyrighted information and republishing it is already illegal.....

Sep 29, 2023 at 2:01pmWeb
Show threadPatrick SamphireSep 29, 2023
@vey981 Still better to stop that list of bots, IMO, even if others continue. But no one has to do it if they don't want to.