dougdougiedougdoug
Spring Jo š„ 
šApple did the research; LLMs cannot do formal reasoning. Results change by as much as 10% if something as basic as the names change.
https://garymarcus.substack.com/p/llms-dont-do-formal-reasoning-and
Not too surprising, but a big weight like Apple standing behind this should shift sentiment more and more
refraction 
@ShadowJonathan we knew this just based on how they arrive at answers. how can you reason if you don't understand the words you're using?
un_pogaz@elexia @ShadowJonathan Because OpenAI and co. marketed them as it. And many, many people swallowed it raw.
The technological realities of LLM have always been secondary in this AI rush, it's all been about marketing, and Oh boy has it worked.
It will take a long time and a lot of demonstrations, including the simplest ones, for the world to understand their mistakes and pull out of this impasse. Or the next revolutionary, distributive technology in the Silicon Valley cycle will hijack all the fund.
Now, it's a matter of proving that the ad was misleading, and that is harder than creating a pretty beautiful lie.
Lulu š ××××This is not surprising at all and I don't understand why anyone had to waste time and resources on demonstrating a self-evident fact that was known before the research even started.
Levi@lulu @ShadowJonathan yeah I don't know anything about LLMs really but I thought this was a fundamental. There's no reasoning, they don't understand meaning, they just make sentences by jumbling words together based on probabilities.
felizYes, the problems posed to the #LLMs in this study are mathematical in nature or logic problems ā why are systems that are trained to produce text expected to produce any meaningful results here?
@ShadowJonathan @feliz @lulu are they though? I don't know enough about LLMs and their implementations to refute you but it seems as though anyone who has ever entered a prompt and looked at the response could understand that there is no reasoning.
Yes. That's why so many people called it out for the lie that it is. LLMs are nothing like their marketing. They are not even AI. It's nothing but autocorrect powered by stolen intellectual property and enough energy to destroy our planet. So yeah, very advanced autocorrect (for an unacceptably high price) but not even slightly resembling AI.
Bolt@ShadowJonathan Maybe now that *big tech company* not *independent, well-respected researchers* has told companies what they need to know, maybe they'll actually listen?
Maybe?
Mike Taylor š¦
yopp@ShadowJonathan itās really weird that some people are pushing LLM as something that can reason, while its architecture is Key-Value storage with sophisticated probabilistic query and value encoding mechanisms.
LLMs just donāt have enough layers for anything besides queries, so it canāt have any relational capabilities that allow to make multi step decisions.
Also tokenization hides a lot of structure of the language from encoding process, which adds additional source of errors.
Iām sure we can build something that can reason at some point, but it requires very diffirent and more complex architecture.
Novaš§āØ@alex @ShadowJonathan it's definitely not key-value storage, it's more latent space interpolation than anything... tokenization isn't that bad either, it's mostly just that there's no depth like you mentioned!
@technobaboo @ShadowJonathan but isnāt latent space interpolation just basically maps some input to some output? Input embeddings always return the same next most probable token? Sure, underlying structure is more complex with all this encodings, decoding and attention layers, but on a surface you just putting some query in and getting some value out from the constant storage. And we have no idea how to reliably write or read from that storage, but thatās entirely diffirent problem :D
@alex @ShadowJonathan it's definitely a mapping function for sure but key value storage implies a direct mapping via table when it's way less linear than that
Andreas K@ShadowJonathan Next up, water is wet. Surprising research by Apple. If only Apple did their research earlier, the Oceangate disaster could have been prevented.
@ShadowJonathan if they canāt reason then they arenāt sapient
dianaisnthere@ShadowJonathan its almost like they just string words together without knowing what they mean..
Kay Ohtie š FWA@ShadowJonathan seeing some folks in the replies on the article, they're not responding to the assertions of the article, but instead about it more broadly, and making incorrect ones there too. "Occasional error is not fatal" but when I ask something to summarize an article, if it grossly misrepresents the article, why should I have to rerun it to make sure it's accurate without me reading it first? "Um but it's good at other things!" The article didn't state otherwise, it was specifically about reasoning, there is no need for the defensiveness.
Anders Puck Nielsen@ShadowJonathan Why would we judge LLMs on their ability to solve complex tasks? The interesting thing is if they can solve simple tasks well enough to be useful.
purple: licenced to sticky plaster@anderspuck @ShadowJonathan it was given a simple task, and failed
Bee O'Problem 
@anderspuck @ShadowJonathan because they're being sold as if they can solve complex tasks
LLMs can use a prompt to generate text based off of a huge pile of content produced by other people. Sometimes that text is an exact copy of the original text. They may "solve" a problem if the solution is contained in their training data and your prompt is able to retrieve it.
They're a (very) improved version of a Markov chain. Not a problem solver of any sort
Cassandrich@anderspuck @ShadowJonathan Which they also can't do.
@dalias @ShadowJonathan They can absolutely do certain things well enough to be useful. Create a fairly accurate transcript of a podcast, for example.
JWcph, Radicalized By Decency@anderspuck @dalias @ShadowJonathan Weeeeeeelllll.... "fairly" & "useful" are pretty load-bearing here - like, yes, they can, but they still make the sort of errors that completely change the meaning of the content & there's no way to check for it except human proofreading, which itself is unreliable at low-cost scale (i.e. a non-specialist low-paid worker checking many texts at a fast pace). Suffice to say that even for this, LLMs are wildly oversold.
@jwcph @dalias @ShadowJonathan I used an LLM to create a first draft of the transcript here, for example. Without that help there just wouldnāt be any transcript because it would take too much time. So that for me is definitely in the category of āusefulā.
https://www.logicofwar.com/why-did-experts-fail-to-predict-russias-invasion-of-ukraine/
Why did experts fail to predict Russia's invasion of Ukraine?
Hello, In this video, I discuss why so many experts failed to accurately predict the Russian invasion of Ukraine in 2022. Most experts at the time were saying that it was very unlikely that Russia would invade Ukraine. Of those who did foresee an invasion, many dramatically overestimated the capabilities
@anderspuck @dalias @ShadowJonathan Sure - now all you have to figure out is how much you'd pay for that usefulness, because this is only happening to become an extremely lucrative business for somebody.
(no, that's not a different topic; the problem complex here is functionality + usefulness + environmental impact + business model)
@jwcph Letās see how it develops. Ollama is working great for me, but it does require a fairly good computer. So yes, either taht processing power has to be local with the user or somewhere centralized.
Jeff Grigg@anderspuck @dalias @ShadowJonathan
LLMs are NOT doing *speech to text* translation -- doing transcripts from audio (podcast). That's a different set of AI technologies.
The industry has been developing "AI" technologies since before I was born. Some are quite useful.
It's the "Generative AI" subset (which includes LLMs, chatbots) that is so misleading, mostly useless, and incredibly wasteful.
@JeffGrigg @anderspuck @ShadowJonathan This. š The industry is all about muddling these differences so they can use the utility of one thing to justify a different piece of garbage they want to sell.
@JeffGrigg @dalias @ShadowJonathan True. I kind of bundled ChatGTP and Whisper in that statement.
I donāt find generative AI useless, though. There are many tasks for which it is very good, but probably not those flashy ones many people are thinking about. For example an LLM is much better at sentiment analysis than older methods.
I donāt find generative AI useless, though. There are many tasks for which it is very good, but probably not those flashy ones many people are thinking about. For example an LLM is much better at sentiment analysis than older methods.
@anderspuck @JeffGrigg @ShadowJonathan Are you sure about that? I'm pretty sure they do an extremely racist version of "sentiment analysis".
Amber 
@dalias @anderspuck @JeffGrigg @ShadowJonathan Letās be clear, the LLM is not developing racism out of nowhere. It is just able to amplify racial bias in the dataset. The stuff used to train it was already racist. Itās extremely hard to filter that out. I still laugh at tip culture being ingrained into LLMs. Some would do "better work" than normal if you bribed it with a tip. Freaky stuff.
mirabilos@dalias @ShadowJonathan @anderspuck no, never reliable enough. This stems from how they are designed.
They are incapable of asking for help if they donāt understand a passage, for example, writing down something hallucinated* instead.
*) Iām aware that this is not a good term to use for this but I donāt have a better one handy before coffee.
SouprMatt@anderspuck @ShadowJonathan š¤” Sure, letās build a nuclear power plant to give enough power for this AI to do something simple a drunk human could do at 3am on just the power of a half-bag of Cheetos. LLM is crap, and wastes obscene amounts of water and fossil fuels to power it. Everybody should just back away from it, slowly until it dies.
@anderspuck because they're expected to solve complex tasks, they're being sold as if they can solve complex tasks, and that they have a fail and error rate enough that they're not safe
They want these things to drive cars and make decisions that involve human lives.
Softwarewolf@ShadowJonathan @anderspuck Not to mention they are insanely expensive to operate. The cost-to-benefit ratio is not sustainable, even for most of the tasks they *can* do.
@faoluin @ShadowJonathan Isnāt that more a question about green energy transition than about LLMs as such?
@anderspuck @ShadowJonathan No, it doesn't matter what kind of energy they're consuming, because energy always has a cost to produce, and again the cost-to-benefit ratio isn't there. LLMs are creating scarcity for relatively little actual positive benefit.
It's also not strictly about power; the same arguement applies to water consumption as well.
kasperdThere is a huge difference between LLM and the automation used for cars. How one of the two behaves cannot be used to draw any conclusions about the other.
@kasperd @anderspuck @ShadowJonathan
Dittos! I was about to post the same thing.
The industry has been developing "AI" technologies since before I was born. Many work quite well, and are useful. Some save money. Some save lives.
You probably interact with "traditional" AI systems far more often than you realize.
Each has to be evaluated based on its costs and benefits and risks.
Generative AI / LLMs Chatbots are a dangerous wasteful SCAM.
Self-driving cars are still "iffy."
Self-driving cars are iffy, but human driven cars are dangerous. A self driving car might already be safer than one driven by a human.
The hard question is what will people choose if they are given the choice between two accidents that can be blamed on human drivers or one accident with a self-driving car where there isn't anyone to blame.
Ava 
@kasperd The solution to cars being dangerous isnāt to replace the drivers. Itās to get rid of the cars in as many places as possible to replace them with free public transit, bikes, and other safer and greener options
By all means let's have better public transit. It doesn't need to be free in order to have an effect. How well it functions is a more important factor in getting people to use it than the price.
@kasperd Sure, but if itās free at least you donāt get cops shooting people to death over a $3 fare (as happened in NYC less than a month ago)
I think the real issue to be addressed in that story is not the price of the ticket.
Nini@anderspuck As long as you can define "simple" as it applies to the LLM because some basic stuff it cannot do due to the lack of cognition. When I say that it cannot do these things I mean it needs to be able to do said tasks unaided (as in no further prompts or guiding prompts) with complete accuracy, if it can't do those things then it's not doing the task even if it's partially right.
MOVED TO @@ShadowJonathan not to sound antiintellectual, but isn't it kinda obvious that a *text* generator, no matter how complex, can't do abstract reasoning?
Scott Feeney@halva @ShadowJonathan yeah, I appreciate the demonstrations, but this feels a little like, "New study confirms bicycles cannot fly."
Ruben Schade 
š° š¦šŗCompanies like OpenAI and their defenders claim generative AI can reason, learn, etc. We know itās nonsense, but itās still extremely important it gets called out.
Cybarbie@rubenerd @graue @halva @ShadowJonathan This is a huge problem. They are the experts. Hinton and Ilya claiming a function can have understanding. Why are they lying? Seems counter productive to scare the hell out of people. Well I know why Ilya is lying he just got $1bn...
Marty Fouts@nf3xn @rubenerd @graue @halva @ShadowJonathan I doubt Hinton is lying although heās probably wrong. Thereās a problem in philosophy: is the mind separate from the body? If itās not, then it should be possible to model the brain well enough to simulate thought processes (at least in principle.)
Computational physics tells us that there is a function that could perform the simulation and Hintonās career is looking for it.
@MartyFouts @rubenerd @graue @halva @ShadowJonathan How can he be wrong? He does not understand what he has wrought? He is literally the pioneer. One must assume that they have a far better grasp of how it works than anyone. What you are talking about is a million times removed from these crude devices. We are quantum beings that collapse wave functions (allegedly).
ZĢoĆ©
ShadSterling@MartyFouts @nf3xn @rubenerd @graue @halva @ShadowJonathan thereās a huge gap between āpossible in principleā and āthis does it nowā. A kite can fly, but being able to build a kite doesnāt mean you can build an airplane
@graue @halva @ShadowJonathan But imagine of the world's most awful people were pouring hundreds of billions of dollars into telling everyone bicycles can fly. Then you would need to spend resources refuting that. š¤¬
enoch_exe_inc@dalias @graue @halva @ShadowJonathan Youād think that people who own a bicycle can just checkā¦
On a tangentially related note, flying bicycles are invented by future humanity āThe Dark Forestā personal flying vehicles in the form of helicopter backpacks. Theyāre ābicyclesā in the sense that theyāre two counter-rotating coaxially-mounted propellers. Thatās actually not a bad idea. If only we poured billions of dollars into making that work.
@enoch_exe_inc @dalias @graue @halva
> Youād think that people who own a bicycle can just checkā¦
does the emperor have no clothes? would people call him out on it?