Spring Jo 🥚 
🍀Apple did the research; LLMs cannot do formal reasoning. Results change by as much as 10% if something as basic as the names change.
https://garymarcus.substack.com/p/llms-dont-do-formal-reasoning-and
Anders Puck Nielsen@ShadowJonathan Why would we judge LLMs on their ability to solve complex tasks? The interesting thing is if they can solve simple tasks well enough to be useful.
@anderspuck because they're expected to solve complex tasks, they're being sold as if they can solve complex tasks, and that they have a fail and error rate enough that they're not safe
They want these things to drive cars and make decisions that involve human lives.
Softwarewolf@ShadowJonathan @anderspuck Not to mention they are insanely expensive to operate. The cost-to-benefit ratio is not sustainable, even for most of the tasks they *can* do.
@faoluin @ShadowJonathan Isn’t that more a question about green energy transition than about LLMs as such?
@anderspuck @ShadowJonathan No, it doesn't matter what kind of energy they're consuming, because energy always has a cost to produce, and again the cost-to-benefit ratio isn't there. LLMs are creating scarcity for relatively little actual positive benefit.
It's also not strictly about power; the same arguement applies to water consumption as well.