Prof. Emily M. Bender(she/her)A thought experiment in the National Library of Thailand—or why #ChatGPT (or any other language model) isn't actually understanding.
Osma Suominen@emilymbender
Thank you for this enlightening thought experiment! This is like Searle's famous Chinese Room example, except that it aptly describes modern LLMs instead of old rule-based AI systems.
Thank you for this enlightening thought experiment! This is like Searle's famous Chinese Room example, except that it aptly describes modern LLMs instead of old rule-based AI systems.
Bornach@osma @emilymbender
The difference is that the Chinese Room has a big book of instructions telling you how to create a response using Chinese characters. The Thai library doesn't even have that and you must somehow write this big book of instructions yourself.
I imagine archaeologists discovering a library of a lost civilization who had figured out how to communicate with an alien race. This alien race has now transmitted a question to modern humans. How do we formulate a response?
@osma @emilymbender
Also missing is an analog of Reinforcement Learning with Human Feedback and the hidden economy of low paid gig workers labelling the books in the Thai library. We'd have to imagine there was computer in the library that had a broken monitor, so all it did was beep annoyingly when you entered something it didn't like.
Also missing is an analog of Reinforcement Learning with Human Feedback and the hidden economy of low paid gig workers labelling the books in the Thai library. We'd have to imagine there was computer in the library that had a broken monitor, so all it did was beep annoyingly when you entered something it didn't like.
@osma @emilymbender
And a key flaw of these thought experiments is that we still can assume sentient being communicating with another sentient being and having a lot in common - needing to eat, having to ask for things of others, forming collectives, motivation vs instinct. I thought this scene in Arrival was most enlightening:
But how does a LLM on a computer that has never had to beg to be provided with electricity understand the concept of a child asking for food?
