Apple did the research; LLMs cannot do formal reasoning. Results change by as much as 10% if something as basic as the names change.

https://garymarcus.substack.com/p/llms-dont-do-formal-reasoning-and

LLMs don’t do formal reasoning - and that is a HUGE problem

Important new study from Apple

Marcus on AI
Not too surprising, but a big weight like Apple standing behind this should shift sentiment more and more

@ShadowJonathan Maybe now that *big tech company* not *independent, well-respected researchers* has told companies what they need to know, maybe they'll actually listen?

Maybe?