My daughter, who has had a degree in computer science for 25 years, posted this observation about ChatGPT on Facebook. It's the best description I've seen:

@DrewKadel Does that image have alt text? I don't see the usual tooltip that I expect to pop up over an image...

Anyway, I like the response. I also think it's worth keeping in mind that (many) people are hoping this line of machine learning research is going to lead to a system that *does* give real answers, or at least is good enough at completion that its responses are indistinguishable from real answers. So there's always going to be a drive to test how well the models are doing and to push them toward giving more realistic and reliable responses, regardless of the fact that they're not actually designed to do that.

@diazona @DrewKadel No alt-text, and as a screen-reader user, I now don't know what the poster's daughter actually said, unfortunately...

@FreakyFwoof @diazona @DrewKadel

Here you go,
"Something that seems fundamental to me about ChatGPT, which gets lost over and over again:
When you enter text into it, you're asking "What would a response to this sound like?"
If you put in a scientific question, and it comes back with a response citing a non-existent paper with a plausible title, using a real journal name and...

@FreakyFwoof @diazona @DrewKadel

...an author name who's written things related to your question, it's not being tricky or telling lies or doing anything at all surprising! This is what a response to that question would sound like! It did the thing! But people keep wanting the "say something that sounds like an answer" machine to be doing something else, and believing it *is* doing something else. It's good at generating things that sound...

@FreakyFwoof @diazona @DrewKadel like responses to being told it was wrong, so people think that it's engaging in introspection or looking up more information or something, but it's not, it's only, ever, saying something that sounds like the next bit of the conversation. "
@simon_lucy @diazona @DrewKadel Profound, and oh so true.