popcar2Nvidia Announces DLSS 5, and it adds... An AI slop filter over your game
orcaEven if it looked good, it has zero context of the original artists’ intent. This is like having AI summarize pages of a book. You’re now locked a layer away from the original artist’s work and it’s a layer controlled by corpos. No thank you.
At least 2 layers.
LLMs don’t think. They copy paste something that’s been found repeatedly in the data it was trained on, statistical probability of words going with other words. Hell, it doesn’t even know what words are or much less mean. So it’s at least 2+ layers removed from the truth, one being the one you pointed out, and another being an amalgamation (mishmash) of the data it was trained on.
I get that lemmy hates AI, and I’m not going to try to talk you out of that, but please stop repeating this factually incorrect myth. LLMs are not stochastic parrots, despite what you may have heard. And they do think… to a degree. Note that they’re by no means everything CEOs and tech bros want them to be, but if you’re going to criticize them, please do it accurately.
They do know the meaning of words, but only in relation to other words. It’s how they work. It’s not a statistical thing like word frequency patterns— they’re not doing the same thing autocomplete does. Instead, they’re doing math on words in a several hundred-thousand dimensional array where placement on this grid indicates the meaning of the word— one vector direction indicates plurals, another indicates rudeness or politeness, another indicates frog-like, another might indicate related to 1993 ibm pentium CPUs, etc, etc, etc. It developed this array via training on terabytes of text, but it’s not storing a copy of that text, nor looking it up, nor copying anything from it… it’s defining words based on how they are used, then doing math on it to figure out what is the most appropriate thing to say next— not the most likely thing according to statistics, the most meaningful based on the definitions of the words it understands.
They really do not copy and paste. They do use definitions. They do think about the words in a very real way.
They don’t apply logical consistency and fact checking. There are hacks to make them talk to themselves in a way that following the meaningful definitions of words will more likely lead to fact checking and logical consistency, but it’s not 100% fool proof.
Having a number that relates words to other words is not underatanding words. Stop believing the hype for fuck’s sake. What they ‘know’ is NOT knowledge. They do not know anything. Period.
There is a reason they start to fail when trained on other slop; because they don’t know what any of it means!
Saying that an LLM knows words is not a value judgement. It doesn’t mean “LLMs are sentient” or “LLMs are smart like humans”. It’s doesn’t imply they have real world experiences. It’s just a description of what they do. That word has been used to describe much more basic kinds of information / functionalities of computers already. What makes it so offensive now?
There is a reason they start to fail when trained on other slop; because they don’t know what any of it means!
If you taught children slop at school they would not get far either. Although training LLMs on LLM output is more akin to getting rid of books and relying on what teachers remember to teach the students.
The importance of that weight comes from humans. It is not intrinsic knowledge even after training.
It comes from the llm and not from the outside, that’s what intrinsic means. How is it not intrinsic knowledge? I think you mean to say without humans to read it, an llm’s output holds no inherent value. That is true and nobody is claiming that it does. llms don’t derive pleasure from talking like humans do so the only value llm output has is from the the person reading it.
Their ‘knowledge’ comes from the basic weights of what word is most likely to follow. It is pure association, and not association like you or I do word association.
llm weights are anything but basic, but regardless, this is also true and lunnrais said as such:
They do know the meaning of words, but only in relation to other words.
The difference between human knowledge and llm knowledge is that an llm’s entire universe is words while humans understand words in relation to real world experiences. Again, nobody is claiming those two understandings are equivalent, just that they are they exist.
Also on the point of statistics, I think the way people understand statistics and the statistics used in llms are vastly different. It is true that an llm finds the which word is most likely to be next, but how it does that is not a classical statistical method. An llm itself is a statistical model, one different than any other statistical model people are familiar with. When one says an llm ‘knows’ or ‘understands’ they mean it has captured abstract information in a incomprehensibly complex digital neural network like how humans capture knowledge in a incomprehensibly complex organic neural network. How it can only use that information for word statistics doesn’t change that it has captured the information
Even simply using the word “know” is anthropomorphising them and is wholly incorrect.
Computers have been getting anthropomorphised for a long time. Why is it only when talking about llms that you start clutching your pearls about it? Why do you think that verb has to be exclusive to humans? To me that seems like a strange and inconsequential thing to dig your heels in.
Your eagerness to fool yourself is beyond sad.
Why is it only when talking about llms that you start clutching your pearls about it?
I am of the opinion now, and this is entirely AI’s fault, that for the collective mental health of our society, a grocery store self-checkout should not even be allowed to “thank” you for your purchase.