My fellow journalists, I need you to stop talking about the Turing Test; it was never a good metric. Turing was a computer scientist, not a psychologist. ELIZA passed it, and also was a decent Person-centered therapy bot. Some people actually used it that way, even knowing it was a bot.
A) There are many paths to the top of a mountain, and
B) LLMs and other AIs aren't real girls and boys just because we feel they are.
Eliza never passed the imitation game and wasn't structured to plausibly take it.
I don't think the current language models are quite up to that particular test either (certainly not with the default settings), but they're in the ballpark. It is certainly true that the limits of a purely linguistic test have become transparent of late.
It's a far better test than chess playing, certainly. On the other hand, Goodhart's law applies.
Quinn Norton
GLC