My daughter, who has had a degree in computer science for 25 years, posted this observation about ChatGPT on Facebook. It's the best description I've seen:

@DrewKadel Does that image have alt text? I don't see the usual tooltip that I expect to pop up over an image...

Anyway, I like the response. I also think it's worth keeping in mind that (many) people are hoping this line of machine learning research is going to lead to a system that *does* give real answers, or at least is good enough at completion that its responses are indistinguishable from real answers. So there's always going to be a drive to test how well the models are doing and to push them toward giving more realistic and reliable responses, regardless of the fact that they're not actually designed to do that.

@diazona @DrewKadel No alt-text, and as a screen-reader user, I now don't know what the poster's daughter actually said, unfortunately...

@FreakyFwoof @diazona @DrewKadel

Here you go,
"Something that seems fundamental to me about ChatGPT, which gets lost over and over again:
When you enter text into it, you're asking "What would a response to this sound like?"
If you put in a scientific question, and it comes back with a response citing a non-existent paper with a plausible title, using a real journal name and...

@FreakyFwoof @diazona @DrewKadel

...an author name who's written things related to your question, it's not being tricky or telling lies or doing anything at all surprising! This is what a response to that question would sound like! It did the thing! But people keep wanting the "say something that sounds like an answer" machine to be doing something else, and believing it *is* doing something else. It's good at generating things that sound...

@FreakyFwoof @diazona @DrewKadel like responses to being told it was wrong, so people think that it's engaging in introspection or looking up more information or something, but it's not, it's only, ever, saying something that sounds like the next bit of the conversation. "
@simon_lucy @diazona @DrewKadel Aah, thanks very much. I appreciate it.
@FreakyFwoof @diazona I just woke up and added the alt text
@DrewKadel @diazona Thank you for doing so. It does help.
@DrewKadel @diazona And as a result, I feel confident in sharing it. As a screen-reader user without any sight whatsoever, it's sometimes hard to know whether what I *think* is true, is true. Without proper context, I could be sharing text that, in a post, says one thing, but the image could well be anything, from hate-speech to well, you can imagine. Alt-text isn't just for Christmas, it helps so many, many people.
@diazona @DrewKadel But why? What would be the purpose of a machine learning system that does give good answers?

@newrambler @diazona @DrewKadel

The purpose is to gain attention. The Turing Test was shattered when ChatGPT accused a real lawyer with a fake crime.

@j7748283 @newrambler @diazona But real people defame real people all the time! Look at Sidney Powell, Fox News and Dominion election systems!
@newrambler @DrewKadel It'd be like having a personal research assistant, I guess - it could save you some time searching the internet for random facts, and depending on what integrations it has, it could handle things like checking your schedule (or someone else's schedule, when you need to meet with them), finding times when you can go see a movie, looking up directions, ordering food, etc. Not that any of this seems especially high-impact, but it *does* make a real difference when mundane tasks become simpler and quicker.
@diazona @newrambler There are possibilities for sophisticated text generation and research in companies etc. With excellent operators and supervision it could do white papers, reports etc. But rolling it out as a $10k/mo subscription for business wouldn't get much action or attention so they market to a naive public.

@DrewKadel @diazona I research and write for a living, so I'm likely biased here. I can see the scheduling thing being useful, but how do you get a bot to do good research when it is prone to making up citations to articles that don't exist?

There's so much terrible writing out there that I suppose having machines write things won't make much of a difference in the white papers full of stakeholders and proactive solutions and so on.

@DrewKadel @diazona And I suppose if I get replaced by ChatGPT, I can go work at the P&G plant down the street.
@newrambler @DrewKadel We're talking about a (hypothetical) bot that gives good answers, which means - among other things - it will not make up citations to articles that don't exist.
@diazona Alt text added this morning
@DrewKadel Thanks! I can confirm I see the tooltip now.