Patrick SamphireSep 29, 2023

To stop AIs scraping your website for content, add this to your robots.txt file on your website.

Thanks to Neil Clarke for most of these.

User-agent: CCBot
Disallow: /

User-agent: ChatGPT-User
Disallow: /

User-agent: GPTBot
Disallow: /

User-agent: Google-Extended
Disallow: /

User-agent: Omgilibot
Disallow: /

User-Agent: FacebookBot
Disallow: /

Show threadRachael L
@patricksamphire @erikvorhes I also likely uselessly added language to my copyright notice that I do not consent to have my pages used in generative AI training models. Uselessly since I don’t imagine it would be useful unless I wanted to sue or something and maybe not even then. But at least I feel declaring non-consent is culturally useful.
Sep 29, 2023 at 1:15pmWeb
Show threadPatrick SamphireSep 29, 2023
@r343l @erikvorhes Good idea.