Mastodawn

Seirdy Aug 6, 2023

If you blocked ChatGPT-User in your robots.txt, you need to update it. OpenAI now uses the “GPTBot” user-agent.

User-agent: GPTBot
Disallow: /

Got 100 hits yesterday. Ugh.

Edit: this won’t stop them in their tracks but it will at least make things slightly less convenient for OpenAI.

Blocking certain bots

I don’t want my content on those sites in any form and I don’t want my content to feed their algorithms. Using robot.txt assumes they will ‘obey’ it. But they

Seirdy’s Home

Show thread

Jens Oliver Meiert Aug 7, 2023

@Seirdy, everyone, do you have a source? OpenAI docs still seem to refer to “ChatGPT-User” (as on https://platform.openai.com/docs/plugins/bot).

(Another question: The docs refer to users and plug-ins. Is there a reference that OpenAI uses only one bot for scraping?)

OpenAI Platform

Explore developer resources, tutorials, API docs, and dynamic examples to get the most out of OpenAI's platform.

Show thread

Seirdy

@j9t See https://platform.openai.com/docs/gptbot

OpenAI Platform

Explore developer resources, tutorials, API docs, and dynamic examples to get the most out of OpenAI's platform.