LeelooFeb 25

As a software developer who took an elective in neural networks - when people call LLMs stochastic parrots, that's not criticism of their results.

It's literally a description of how they work.

The so-called training data is used to build a huge database of words and the probability of them fitting together.

Stochastic because the whole thing is statistics.
Parrot because the answer is just repeating the most probable word combinations from its training dataset.

Calling an LLM a stochastic parrot is lile calling a car a motorised vehicle with wheels. It doesn't say anything about cars being good or bad. It does, however, take away the magic. So if you feel a need to defend AI when you hear the term stochastic parrot, consider that you may have elevated them to a god-like status, and that's why you go on the defense when the magic is dispelled.

Show threadWolf480plFeb 26
@leeloo on the flipside, I feel like some people use the term "stochastic parrot" or "it just completes the next token" to imply that "therefore it cannot be intelligent" - is that correct reasoning?
Show threadLeelooFeb 26
@wolf480pl
Of course it can not be intelligent, it's just a huge database of probabilities.
Show threadWolf480plFeb 26

@leeloo pretty sure that's a fallacy, kinda like "a sculpture is just stone, therefore it can't be beautiful", or "a cell is just a bunch of proteins, therefore it cannot be a living creature".

Now, I'm not saying a huge database of probabilities can be intelligent (I hope it can't), just that I think a better argument is needed why in the case of a database of probabilities, what it's made of prevents it from being intelligent.

Show threadJohn Tinker
@wolf480pl @leeloo
The effect that you are noticing is because the writers of the training material were intelligence. You are seeing the reflection of their intelligence in the output of the LLM: Here is output from an LLM that describes what an LLM is, and what it is not: https://johntinker.substack.com/p/misunderstanding-as-a-commutator
Misunderstanding the Chatbot, seen as a Commutation Operator

There appears to be a general misunderstanding that the large language model is a form of artificial general intelligence.

John’s Substack
Feb 27 at 3:59pmWeb