Matti Airas
Drew KadelMy daughter, who has had a degree in computer science for 25 years, posted this observation about ChatGPT on Facebook. It's the best description I've seen:
Nate Gaylinn@DrewKadel Love this. We want so badly for ChatGPT to produce answers, opinions, and art, but all it can do is make plausible simulations of those things. As a species, we've never had to deal with that before.
ani betts@ngaylinn @DrewKadel to be fair, most people on the Internet given a question, write the right answer rather than everyone consistently writing the same wrong answer!
(* For most things, I'm sure there are several compelling counterexamples)
@anizocani @DrewKadel Oh, totally. Often the most statistically likely response is also the correct response. Just... not always.
Maltimore@ngaylinn @anizocani @DrewKadel
I wish more people would understand this.
That's why I really dislike the take that it's "just fancy autocomplete". In order to have a *really* good autocomplete, you'd have to in one way or another internalize much of the world's knowledge. So, "just fancy autocomplete" is really not the witty criticism some think it is.
Mark Otway@maltimore
The reason we call it spicy autocomplete is because it is just a prediction model on the stuff it's ingested. What you seem to miss is that we already had all this, with Google. All LLMs have done is make it feel like something is answering, instead of being honest and returning results that match a search query.
@markotway @maltimore @ngaylinn @DrewKadel babe. it's two years later. you've been in a coma ever since the accident.
Fabian Transchel@ngaylinn @DrewKadel Well we had - conceptually this is *exactly* the same as with Eliza - just two orders of magnitude more sophisticated and two orders of magnitude more connected.
At its core it's really just people lacking technical understanding hallucinating an antropomorphization of a conditional probability distribution.
With ChatGPT, the interface is the innovation, *not* the model.
@ftranschel @DrewKadel Oh? What makes the ChatGPT interface so innovative? :)
@ngaylinn @DrewKadel Using LLMs interactively is surely an innovative way of using them.
Case in point: When reduced to token completion (which ChatGPT hides away), the magic goes away... quite fast (:
@ftranschel @DrewKadel That's certainly true. They do a lot of "theater" to make it seem like you're talking to someone.
Donald Hobern@ngaylinn @ftranschel @DrewKadel
Theatre is an excellent description. There's clearly NLP in front of the prediction engine itself, and some application of post-predictive review steps. But it's all presented as though this composite application is the LLM. It's like attending a seance.
@dhobern @ngaylinn @DrewKadel I agree with your perspective. For me it's more like a computer version of the Oracle of Delphi, but that's essentially just a different representation of the same idea.
Stephen
FreebirdFrolic@ngaylinn @ftranschel @DrewKadel
Is the 'man behind the curtain'
Innovative?
IMHO; it's all more smoke and mirrors to discombobulate an already gaslit world.
Derek McAuley
EQThink you need to invent a new word, "orders of magnitude" doesn't quite cover it.
Bornach@ngaylinn @DrewKadel
Actually we as a species have had to deal with that before.
We call them grifters.
@bornach @DrewKadel I dunno. A grifter still wants something from you. They're hiding their intent, but it's something you can imagine, understand, and detect if you're paying attention.
LLMs are unreadable and unpredictable because they have no intent. They may switch between friend and grifter depending on what sounds right in the flow of the conversation, without any conception of what's good or bad for you or for them.
On the other hand, if a grifter asks an LLM to write them script to achieve something specific, that's another thing entirely...
Dr. Angus Andrea Grieve-Smith@ngaylinn @bornach @DrewKadel They have been specifically marketed as question-answering and search engine tools. The people misrepresenting them are the grifters.
John Private 🛩 🚁 🚀 🛸@grvsmth @ngaylinn @bornach @DrewKadel
Agree - this is a tool that can be used by grifters. It seems like ChatGPT is a relatively trivial problem, generate responses that fit the pattern of language found in authoratative sources. I believe that Open AI is using it as a demonstration, and generate lots of free media coverage to get paying customers to buy their product. The grift is the misrepresentation.
@ngaylinn
LLMs are never your friend
They are always in what can best be described as "the grifter" mode. The entire training regime of a generative AI chatbot is geared towards getting one thing, an upvote from a human rating the quality of the conversation
Admittedly this is an over-simplification. Reinforcement Learning with Human Feedback involves training a reward policy - a 2nd neural network that is ultimately responsible for rewarding the chatbot for giving "good" responses.
Ype Kingma
gordan knott
@GordanKnott @ngaylinn @DrewKadel
Yet another flawed benchmark in which the LLM very likely memorised the answers
https://wandering.shop/@janellecshane/110104164829618120
Without any knowledge of how much the training dataset was contaminated by the medical exam questions/answers (and OpenAI's own whitepaper admits there is contamination)
https://youtu.be/PEjl7-7lZLA?t=4m0s
we cannot really know how it would perform in the real world if say a novel virus were to start spreading
Janelle Shane (@[email protected])
Attached: 1 image Remember seeing something about GPT-4 doing well on standardized tests? It turns out it may have memorized the answers. https://aisnakeoil.substack.com/p/gpt-4-and-professional-benchmarks #gpt4 #AIHype #ThisIsWhyWeDontTestOnTheTrainingData
Ryan Egesdahl 
@ngaylinn @DrewKadel This is why I’m so tired of people calling machine learning “AI”. True AI doesn’t exist, and even if it did, there’s no reason to believe we can or should use it as a tool. ChatGPT is machine learning being used outside of its useful domain and media hyping it beyond credibility.
@deriamis @ngaylinn The "AI" label is misleading, but I'm pretty confident that we will never get to true Artificial General Intelligence - this is only a mirage of being close. It might well be that it could be enhanced to parse questions & redirect queries to best tools & use its LLM to make it sound smooth & chatty, or something like that- so that you'd have a generally useful & accurate research assistant, but not artificial intelligence. We may be stuck with the label however.
@DrewKadel @ngaylinn I suppose I should clarify that I think there exist strong and weak definitions of AI. The strong definition is the one to which you refer, and even if we somehow achieve it, there are moral and ethical concerns surrounding employing it as a tool. The weak definition is just ever more sophisticated ML with no sapience. AI is a “close enough” label where the distinction makes no difference. My point is that it clearly does for how ChatGPT is often being used.
@deriamis @ngaylinn @DrewKadel
Machine Optimization
Sounds like we've been dealing with that in human form as misinformation media.
Like any machine optimization algorithm, it all depends on the fitness function. Maybe that's why algorithms (machine and human) are kept secret, it would break the illusion that ChatGPT is smart, that DALL-E is an artist, that the birdsite isn't biased, that Facebook is helping you, that faux news is truthful...
Maddogeco@ngaylinn @DrewKadel I’ve been asking both it and bard to write a small piece of code for a iot project I’m working on. The responses look plausible but are wrong. I can’t code so i can’t tell how or why they are wrong. The answers look impressive and 90% functional but they fall at the last step.
PointlessSpike@Maddogeco I've been using ChatGPT for work (as a programmer) for a while. Yeah, you can never just trust it. It's great if you need to change how something works, it's great for generating test data, or writing SQL statements, but for code you need to learn how to use it. People think you can use it for everything and that's just not how that works.
William B Peckham@ngaylinn @DrewKadel So, you have never had to ride herd on 8 year olds?
Scott Galloway@ngaylinn @DrewKadel Have you met religion? Literally inventing stories to explain reality.
William Pietri@scottgal For sure! Back in the olden days if you had questions about the lights in the sky, you had to find somebody with low moral fiber to make up a plausible answer. Now we've built a machine that will instantly give you comforting nonsense. If that isn't progress, I don't know what is.
David Zaslavsky@DrewKadel Does that image have alt text? I don't see the usual tooltip that I expect to pop up over an image...
Anyway, I like the response. I also think it's worth keeping in mind that (many) people are hoping this line of machine learning research is going to lead to a system that *does* give real answers, or at least is good enough at completion that its responses are indistinguishable from real answers. So there's always going to be a drive to test how well the models are doing and to push them toward giving more realistic and reliable responses, regardless of the fact that they're not actually designed to do that.
Andre Louis@diazona @DrewKadel No alt-text, and as a screen-reader user, I now don't know what the poster's daughter actually said, unfortunately...
Simon Lucy@FreakyFwoof @diazona @DrewKadel
Here you go,
"Something that seems fundamental to me about ChatGPT, which gets lost over and over again:
When you enter text into it, you're asking "What would a response to this sound like?"
If you put in a scientific question, and it comes back with a response citing a non-existent paper with a plausible title, using a real journal name and...
@FreakyFwoof @diazona @DrewKadel
...an author name who's written things related to your question, it's not being tricky or telling lies or doing anything at all surprising! This is what a response to that question would sound like! It did the thing! But people keep wanting the "say something that sounds like an answer" machine to be doing something else, and believing it *is* doing something else. It's good at generating things that sound...
@FreakyFwoof @diazona @DrewKadel like responses to being told it was wrong, so people think that it's engaging in introspection or looking up more information or something, but it's not, it's only, ever, saying something that sounds like the next bit of the conversation. "
@FreakyFwoof @diazona I just woke up and added the alt text
@DrewKadel @diazona Thank you for doing so. It does help.
@DrewKadel @diazona And as a result, I feel confident in sharing it. As a screen-reader user without any sight whatsoever, it's sometimes hard to know whether what I *think* is true, is true. Without proper context, I could be sharing text that, in a post, says one thing, but the image could well be anything, from hate-speech to well, you can imagine. Alt-text isn't just for Christmas, it helps so many, many people.
Laura Crossett@diazona @DrewKadel But why? What would be the purpose of a machine learning system that does give good answers?
@newrambler @diazona @DrewKadel
The purpose is to gain attention. The Turing Test was shattered when ChatGPT accused a real lawyer with a fake crime.
@j7748283 @newrambler @diazona But real people defame real people all the time! Look at Sidney Powell, Fox News and Dominion election systems!
@newrambler @DrewKadel It'd be like having a personal research assistant, I guess - it could save you some time searching the internet for random facts, and depending on what integrations it has, it could handle things like checking your schedule (or someone else's schedule, when you need to meet with them), finding times when you can go see a movie, looking up directions, ordering food, etc. Not that any of this seems especially high-impact, but it *does* make a real difference when mundane tasks become simpler and quicker.
@diazona @newrambler There are possibilities for sophisticated text generation and research in companies etc. With excellent operators and supervision it could do white papers, reports etc. But rolling it out as a $10k/mo subscription for business wouldn't get much action or attention so they market to a naive public.
@DrewKadel @diazona I research and write for a living, so I'm likely biased here. I can see the scheduling thing being useful, but how do you get a bot to do good research when it is prone to making up citations to articles that don't exist?
There's so much terrible writing out there that I suppose having machines write things won't make much of a difference in the white papers full of stakeholders and proactive solutions and so on.
@DrewKadel @diazona And I suppose if I get replaced by ChatGPT, I can go work at the P&G plant down the street.
@newrambler @DrewKadel We're talking about a (hypothetical) bot that gives good answers, which means - among other things - it will not make up citations to articles that don't exist.
@diazona Alt text added this morning
@DrewKadel Thanks! I can confirm I see the tooltip now.
Patashu (Dragon Style!)It's so important for everyone to understand this
Andre@DrewKadel No introspection and only waiting to generate the next line of a conversation. Sounds like quite a few humans!
@PCOWandre I think that's why so many are fooled, they don't even notice the role of listening, understanding or introspection in thought or communication. Of course they have some., quite a bit really, but they don't value it and ignore it.