Michiel BontenbalJun 12, 2023

Can one determine whether a given text is written by a human or an AI? With the rise of ChatGPT this has become a challenge for teachers the world over.

One possible metric is called ‘perplexity’. Perplexity is about how well a probability model predicts a sample. Large language models have been trained using this metric and since LLM’s have become really good lately, their perplexity score is quite low.

So I set on the idea of testing this.
1/2

#AI #LLM #chatgpt

Show threadMichiel Bontenbal

I have created a Jupyter Notebook and compared 10 examples of AI-generated text with 10 examples of human written text. Using python code I calculated and compared their perplexity.

After just testing 20 examples, we can see in the graph below a clear difference between AI-generated and human written text using their perplexity-score.

This is promising, but lot more testing needed.
#howcomputerslearn
#hoecomputersleren

Repo:
https://github.com/MichielBbal/test_ai_detectors

GitHub - MichielBbal/test_ai_detectors: PoC to detect whether text is AI generated or written by a human

PoC to detect whether text is AI generated or written by a human - GitHub - MichielBbal/test_ai_detectors: PoC to detect whether text is AI generated or written by a human

GitHub
Jun 12, 2023 at 8:01amWeb