"Advances over the past year in the misnamed field of “artificial intelligence” have activated the inverse form of the heuristic that haunts so many disabled humans: most people see the language fluency exhibited by large language models (LLMs) like ChatGPT and erroneously assume that the computer possesses intelligent comprehension — that the program understands both what users say to it, and what it replies."

>>

"Not only are we not close to developing “artificial general intelligence”, we are not even far away from developing AGI, because we haven’t even found a path that could conceivably lead to AGI."

>>

"One thing that particularly seems to lead people astray is the way that ChatGPT gives the impression of “apologizing” in response to exterior challenges. OpenAI’s claim that ChatGPT will “admit its mistakes” is worded to suggest that the algorithm both understands that it has made an error and is in the active process of improving its understanding based on the dialogue in progress."

>>

"Making chatbots that seem to apologize is a choice. Giving them cartoon-human avatars and offering up “Hello! How can I help you today?” instead of a blank input box: choices. Making chatbots that talk about their nonexistent “feelings” and pepper their responses with facial emojis is another choice."

>>

"As a society, we’re going to have to radically rethink when and how and even if it makes sense to trust any information that either originates from, or is mediated by, any kind of machine-learning algorithm — which, if you think about it, currently encompasses nearly All The Things."

>>

As I said -- utterly brilliant. Go read the whole thing!

https://karawynn.substack.com/p/language-is-a-poor-heuristic-for