Ben AdidaFeb 1, 2023

So OpenAI just released a detector of AI-generated text, I assume because of concerns in education / homework.

https://openai.com/blog/new-ai-classifier-for-indicating-ai-written-text/

Maybe this is good?

No, it's very bad.

They claim 26% true positives, 9% false positives. Assume 10% of submitted homework is chatgpt generated, you get the classic counterintuitive outcome of poor predictive power: if a homework is flagged, there's a 3:1 chance it's *human* generated.

This is going to cause a lot of harm. It should be immediately recalled.

New AI classifier for indicating AI-written text

We’re launching a classifier trained to distinguish between AI-written and human-written text.

Show threadfrog
@ben It's even worse. Sooner or later there will be more companies, and they will compete, and you'll have to pay for those that can cheat checkers, and students with rich parents can cheat better. My bet? We'll have to abandon the concept of homework! I just see no way out.
Feb 3, 2023 at 8:57amWeb