In our lab introducing AI writing to students I encourage them to probe the models—to try & find limit cases & places where the “wires” of the machine become visible. Some things they found:

+ the model will refuse to generate a “diss track” or write insults about either specific celebrities or platypuses
+ when asked to rewrite a paragraph “using words without the letter e” it returned the exact same words with all e’s removed

Lab prompt here:

https://s23bbb.ryancordell.org/labs/Lab1-AI/

Lab 1: Amanuenses to AI

> every figure of speech, snowclone, cliché, joke, proposition, statement, and practically every linguistic structure that can be turned into a template is easily explored with a bot. Every work of literature, every writer’s body of work, every literary movement, national literature, and textual corpus is waiting to be analyzed...

A few other random observations (all about ChatGPT, which I didn’t make clear):

3. Attempts to create an SNL skit were fruitless—it did not seem to have SNL in its training data & just generated generic scripts
4. ChatGPT is famously good at synthesis, but it’s often unable to combine critical ideas in new ways. If asked to produce a Marxist analysis of various texts, it often produces one paragraph of Marxist thought & one about the chosen text, but can’t combine the two concepts

It did, however, produce a surprising & quite wonderful Marxist haiku about ratatouille (the dish, not the rodent chef movie) so 🤷‍♂️