kromem

Mapping the Mind of a Large Language Model

https://lemmy.world/post/15661170

Mapping the Mind of a Large Language Model - Lemmy.World

I often see a lot of people with outdated understanding of modern LLMs. This is probably the best interpretability research to date, by the leading interpretability research team. It’s worth a read if you want a peek behind the curtain on modern models.

May 21, 2024 at 10:52pmWeb
Show threadWombleMay 22, 2024
This is a really good science communication article, it describes their work in clear terms (finding structures that relate to abstract concepts, seeing when they are activated and how strengthening and weaking them modifies outputs) and goes intobthe implications for it. Im probably going to save this link as a rebuttal for the people who claim LLMs just predicr the next word and have no concepts embedded in them.
Show threadA_AMay 22, 2024
Yes, good topic, good research…
( you have a few typos : intobthe … into the, predicr … predict, im … i am. )
Show threadmiskMay 22, 2024
I doubt that anyone saying that LLM are calculating next word solely based on previous sequence. It’s still statistics, regardless of complexity.
Show threadWombleMay 22, 2024
Youd be surprised at the level of unthinking hatred around them, but even discarding that Ive seen it said often that LLMs have no internal model of what they are talking about as they are just next word generators. This quite clearly contradicts that interpretation.
Show threadSpedwellMay 22, 2024

concepts embedded in them

internal model

You used both phrases in this thread, but those are two very different thints. It’s a stretch to say this research supports the latter.

Yes, LLMs are still next-token generators. That is a descriptive statement about how they operate. They just have embedded knowledge that allows them to generate sometimes meaningful text.

Show threadricdehMay 22, 2024
Yes, but people forget that our brains, and therefore our minds, are also “simply” statistics, albeit very complex.
Show threadZos_KiaMay 22, 2024
Yeah I found this kind of reductionist talk pushes people to overlook the emerging properties of the system, which is where the meat of the topic is. It’s like looking at a living cell and saying “yeah well this is just chemistry”.
Show threadGeneral_EffortMay 22, 2024
Saying that it’s “statistics” is, at best, unhelpful. It conveys no useful information. At worst, it’s misleading. What goes on with neural nets has very little to do with what one learns in a stats course.
Show threadmiskMay 22, 2024
Most people don’t know what Bayesian statistics are so you could say most people don’t really get how machine learning works in general anyway. It’s not misleading though as it perfectly sets expectations on what you’re getting as output. It’s much more healthy to general understanding of AI than anthropomorphizing very inflexible and limited models.
Show threadGeneral_EffortMay 22, 2024

I would not expect almost human-like conversation on being told that is just statistics. I’d expect something like the old Markov chain jobs. What kind of knowledge leads you to have higher expectations?

Also, how does Bayesian statistics enter into this?

Show threadmiskMay 22, 2024

ELIZA from 1966 was enough to convince people that computer program they were talking to was human. People are now being sold on getting answers to their questions via natural language prompts and those answers are pretty much plausibly sounding sentences that happen to be right sometimes due to probability calculations.

Bayesian statistics is very different from what’s being taught up until high school (at least here) and is foundational to earlier machine learning applications like spam filters. It’s hard to imagine understanding what LLMs do without basics.

Show threadGeneral_EffortMay 22, 2024
Those aren’t the basics, though. That’s how saying it’s statistics is misleading. A Bayesian network is not a neural network.
Show threadSkyeStarfallMay 22, 2024

Yeah, it’s about as useful as saying that all of science is “just statistics”. Which like, in a literal way, it’s true. But science is still what forms the foundation of our entire civilization and base of knowledge.

Knowing that a blood pressure drug works is “just statistics”, but you still take it if your blood pressure is high.

Show threadGeneral_EffortMay 22, 2024
Yes, that’s a valid comparison. It’s worse with neural nets, though. Much of machine learning is literally applied statistics. That is, a program is written that applies statistical methods to data and then adjusts its behavior. So, saying that it’s statistics has the potential to really send people down the wrong track. Many of the “human hallucinations” about AIs result from confusion about this.
Show threadtabarnaskiMay 22, 2024
I think the most interesting thing in this article is the fact that some concepts central to semantics (analogy, connotation) or psychology (bias) kind of emerge naturally in multi layered neural networks of sufficient size. Also that it can sound like different personalities (overconfident, secretive, delusional) if you manipulate the weight or the proximity of features. I’d like to see the same kind of study but for midjourney…
Show threadGiveMemesMay 22, 2024
That’s a chicken and egg situation tho. Is the bias a result of a mind? Or is it the result of being trained on data with common human biases all put together by humans?
Show threadmagic_lobster_partyMay 22, 2024
I would imagine a similar result. Like how the word “cartoon” activates one particular feature. And if you identify this feature you can control the level of “cartooniness” by tweaking the particular feature.
Show threadAkasazhMay 22, 2024
It’s a well written article that raises some good points, however it’s also a bit of an ad for their particular ai, the research is practically only within that LLM and every objection about LLM’s it raises, their model seems to be well adjusted or adjustable to it.
Show threadtechnocritMay 22, 2024
There is no mind. It’s pretty clear that these people don’t understand their own models. Pretending that there’s a mind doesn’t help.
Show threadmagic_lobster_partyMay 22, 2024
You’re reading the title too literally. “Mind” is only mentioned once in the entire article, and that’s in the title.
Show threadDreweliteMay 23, 2024
Ah yes, it must be the scientists specializing in machine learning who don’t understand the model.