Reminder that EVERYTHING Anthropic or OpenAI announce in public is propaganda designed to boost their market cap when they hit the IPO they're aiming for.
It's marketing, folks. There is no intelligence behind the artifice, it's just spicy autocomplete and hucksters in $5000 suits trying to pick your boss's pocket.
https://mastodon.social/@arstechnica/115548040527129225
@menelion @cstross "An autocomplete would never suggest new names, new implementation and architectural ideas, and so on, and so forth."
Why? Autocompletes often make changes I do not want and that are out of context with bad parameters.
LLMs are basically large scale autocomplete, running on GPU, with a grain of stochastic variablity put on top.
@Enthalpiste @menelion @cstross "...a bit of stochasticity on top" this is so far the best description of the "intelligence" in LLMs. Randomness. It does not "invent" -- it picks at random. Similarily it does not "guess" and certainly does not "know".
I find it disturbing when people rely on LLMs to act as teachers for them or ask for advice in legal or medical aspects. Worst of all is that children learn to rely on these services.
A trillion dollar stochastic autocomplete.
@jluukka @menelion @cstross I really had a full "WTF?" moment when I discovered that what some models call "temperature" really is a Boltzmann like "exp(-1/kbT)" sampling over the output probability distribution.
Like... why? This makes sense in statistical physics because of canonical distribution. But I'm not sure there is a solid theoretical ground for this in ML.
Charlie Stross
André Polykanine
G. Clavier
Joel Luukka