The results of this new GSM-Symbolic paper aren't completely new in the world of #AI research. Other recent papers have similarly suggested that #LLMs don't actually perform #FormalReasoning and instead mimic it with probabilistic #PatternMatching of the closest similar data seen in their vast training sets. #GenAI

#Apple study exposes deep cracks in LLMs’ “reasoning” capabilities
https://arstechnica.com/ai/2024/10/llms-cant-perform-genuine-logical-reasoning-apple-researchers-suggest/

Apple study exposes deep cracks in LLMs’ “reasoning” capabilities

Irrelevant red herrings lead to “catastrophic” failure of logical inference.

Ars Technica