LeelooFeb 25

As a software developer who took an elective in neural networks - when people call LLMs stochastic parrots, that's not criticism of their results.

It's literally a description of how they work.

The so-called training data is used to build a huge database of words and the probability of them fitting together.

Stochastic because the whole thing is statistics.
Parrot because the answer is just repeating the most probable word combinations from its training dataset.

Calling an LLM a stochastic parrot is lile calling a car a motorised vehicle with wheels. It doesn't say anything about cars being good or bad. It does, however, take away the magic. So if you feel a need to defend AI when you hear the term stochastic parrot, consider that you may have elevated them to a god-like status, and that's why you go on the defense when the magic is dispelled.

Show threadTobias Ernst
@leeloo The thing is, how can we sure that human intelligence does not essentially work in the same way? My Christian believe tells me we have a soul and LLM's do not, that may be the difference. But from an agnostic perspective, we might reach the point where one cannot tell the difference.
Feb 26 at 11:40pmWeb
Show threadLeelooFeb 26
@tobifant
Not with the current methods, and very lilely not without understanding a lot more about how pur own brains work.
Show threadJames BaillieFeb 27
@tobifant @leeloo Whilst we obviously can't show if humans have a soul, we can absolutely show that humans have e.g. abstracted concept frameworks that are not solely based on averages of language statistics. I understand what an "owl" is, for example, in a way separate to the numerical relationships between the word "owl" and other words. That is a really fundamental information processing difference and allows me to construct *novel* understandings of that concept in ways that an LLM couldn't.
Show threadFrank HeijkampFeb 27

@tobifant
A LLM is not able to reason. It can fool you into believing it is intelligent and self aware, where in fact it just parrots the patterns it has stored. These patterns are however very human-like as they are the result of training on texts written by actual humans.

The fun part starts now where the entire internet got flooded by #ai generated content. All of this will be the training set for the next generation of LLM's. What could possibly go wrong?
@leeloo