Jason GormanWhat makes LLMs work isn't deep neural networks or attention mechanisms or vector databases or anything like that.
What makes LLMs work is our tendency to see faces on toast.
ggdupont@jasongorman
100%
I've been telling this for a while now.
Our tendency for pareidolia goes way beyond just seeing faces. We're desesperately trying to connect other intelligent beings...
100%
I've been telling this for a while now.
Our tendency for pareidolia goes way beyond just seeing faces. We're desesperately trying to connect other intelligent beings...
circfruit@gdupont @jasongorman I didn’t know there’s a word for that 🤨
@circfruit
@jasongorman
There is a large research corpus in human cognition which studies how people think and how our thinking can be (easily) fooled.
It is usually completely ignored by AGI fanboys (so it's easier to make grand claim about artificial cognition)
GLC@circfruit @gdupont @jasongorman
And a nice page for it as well.
https://en.wikipedia.org/wiki/Pareidolia
Pareidolia + theory of mind: mentalist effect
LLM: stochastic parrot
Mix and serve
Peter Middelkoop@jasongorman A thought that slipped trough my mind also.
JWcph, Radicalized By Decency@jasongorman No, seriously - that's exactly what it is & it's a mechanism exploited by charlatans since always: https://softwarecrisis.dev/letters/llmentalist/
Petra van Cronenburg
maya💜@jwcph @jasongorman I’ve always felt like the way chatbots speak was a little too general and assertive for me to believe/trust them, and I’m glad to learn *why* it is like that. The psychic comparison is very effective.
feld@jwcph @jasongorman I've never had a piece of toast with a face on it write hundreds of lines of code for me that actually works because I talked at it tho
@jasongorman @jwcph what do you mean? I tell it that it must pass the test case, pass the formatter, linter, and static analysis... and then it emitted code and executed what I asked and produced a working result that passed the test case
it automatically reads the results of the formatter, linter, compiler, static analysis, and tests cases and corrects any errors it encounters
if you don't give it thorough instructions it will produce garbage. so just give it the same guidance you would a jr dev along with examples of best practices
it automatically reads the results of the formatter, linter, compiler, static analysis, and tests cases and corrects any errors it encounters
if you don't give it thorough instructions it will produce garbage. so just give it the same guidance you would a jr dev along with examples of best practices
@feld @jasongorman Nobody believes you, dude. Let it go.
@jwcph @jasongorman why would I lie? I have literally nothing to gain by convincing someone that you *can* coerce these tools into doing something useful
@feld @jasongorman - and yet you're still trying...
@jwcph @jasongorman do you think spreading these lies will really make these tools disappear or something? what a strange obsession
they'll still be around 5 years from now. they're not going anywhere.
they'll still be around 5 years from now. they're not going anywhere.
Timothy Wolodzko@jasongorman jokes aside, this is a very good metaphor
@tymwol I'm not even sure it's a metaphor 🙂
Infrapink (he/his/him)It's called the ELIZA effect, and we've known about it since 1966: https://en.wikipedia.org/wiki/ELIZA_effect
Pete@Infrapink @jasongorman @tymwol
I think it's also related to the #PeterPrinciple... that people get promoted to a level of incompetence.
The chat part of #ChatGPT is really important because it allows us to correct the LLM's output and give it another chance to completely fool us. When we're satisfied, the output is above our present ability to detect that it is BS.
By "present ability" I mean we might not feel bothered to check the output, or we genuinely might think it's correct.
Óscar Morales Vivó@pete @Infrapink @jasongorman @tymwol It should also be noted that the people most in love with the bullshit-generating machine make a living out of generating bullshit themselves.
Schroedinger@Infrapink @jasongorman @tymwol I think both things apply. One reason that Eliza was so convincing is that it was "mimicking" a psychological therapy session. Where having your ideas repeated back to you is common and a good way of making connection.
It fulfilled peoples expectation of a session. Like faces on toast fulfill peoples expectations. All of the so called "intelligence" is actually in ourselves.
Chat tools can help us work out the answers ourselves sometimes - a well know developers trait. But mostly, it gives us what we expect, and that is good enough for us.
@jasongorman 💯 !
We should stare more often into the sky watching clouds.
We should stare more often into the sky watching clouds.
fromjason.xyz ❤️ 💻*Azure breaks through wall* did somebody say the cloud?
Krypt3ia@jasongorman Digital Jesus toast.
DebC (She/Her)♿️☮️@krypt3ia @jasongorman I'm going to have to steal this phrase.👹🍞
@DebC @jasongorman Approved
Cassandrich@krypt3ia @jasongorman My toast just gets Calvary not Jesus. 😂
@dalias @jasongorman Chinese knockoff
ꓤɔᴉʇɐʇS@jasongorman A streamer I watch has a bot that's literally just a random number generator picking from a set list of predefined phrases that runs on a 10 minute timer or when prompted by chat and you wouldn't believe how shockingly often its totally on point "reacting" to what's happening on screen or responding to chat. So much so that we've been joking it is actually sentient.
Actual chatgpt which that streamer used before that had less hits than the literal RNG list on a timer.
Duco
Stephen Bannasch (316 ppm)Pareidolia is fun and cool unless you are fooling yourself that the faces you see are real and they are talking to you.
Efi (nap pet) 🦊💤@jasongorman if it looks like a duck and it quacks like a duck, it can't possibly be an artificial duck
William Pietri@jasongorman Yes! This plus what I think of as the Stone Soup Effect. A claim of magic gets people to try it, but they don't notice that they're the ones doing most of the value-creating work.
Dec.tar.gz@williampietri @jasongorman
"Finding books in babbling brooks"; that's either Shakespeare or Benford.
"Finding books in babbling brooks"; that's either Shakespeare or Benford.
Yvo Verschoor@jasongorman just as the “I want a lollipop” quote from the last #securitynow podcast*, this is also a nice piece of wishdom for Leo la Portes ( @leo ) quotes collection. I think 😇.
* = see page 15 of: https://www.grc.com/sn/SN-1027-Notes.pdf
* = see page 15 of: https://www.grc.com/sn/SN-1027-Notes.pdf
Max Leibman@jasongorman @purplepadma Right on. As I’ve said so many times, I hate the term “hallucination” because it implies there is something in there that perceives and that therefore can hallucinate. In fact, the only hallucination is happening on our side of the keyboard.
Allan@jasongorman oooh that’s good.
infryq@jasongorman unfortunately, I feel like our tendency to see faces on toast is also what makes human society work
Norm Tovey-Walsh@jasongorman And staggering amounts of theft, of course.
MattChippytea
T'Cyn of Vulcan 🇨🇦🏳️⚧️Faces on toast. Faces. That’s what it actually says. Faces on toast. Faces.
Darn my aging eyes!
kolya
D2@jasongorman nice. Brilliant phrasing vs. we *are* pattern-seeking machines.
Arnim Sommer 🇪🇺@jasongorman
Well, computers are doing something that we interpret as mathematics…
Well, computers are doing something that we interpret as mathematics…
Tom Stoneham@ArnimRanthoron @jasongorman
We have good theories explaining how syntactic rules can implement semantic rules. In fact, nearly 200 years of formal logic.
We also have good theories explaining why a next token generator does not implement linguistic understanding. But sadly these are not well understood outside linguistics and philosophy of language.
Tattie@jasongorman are you implying that Toastface isn't real?
Don't listen to him, Toastface.
jnpegay, emphasis on the “gay”@Tattie @jasongorman my favorite member of wu tang
Paul SomeoneElse
vruzAlso basic ignorance about how technology works.
Never to be underestimated, except by people who don't understand technology.
Raymond Russell@jasongorman
LLM's are doing an automated Mentalism act.
LLM's are doing an automated Mentalism act.
Earthshine@jasongorman I mean if we're really getting down to it, one might argue rather that LLMs just don't work, but our tendency to see faces on toast is what makes people able to sell them anyway.
Weekend Editor"The fault, dear Brutus, lies not in our stars but in us."
We are the ones hallucinating rationality. (In our defense, LLMs pretty much optimize for plausibility.)