Has it ever been good at mathematical/logical problems? It seems it’s good at text-based problems like imitating a writing style or even writing code, but if you ask it a logic puzzle like “if two cars take 3 hours to reach NYC, how long will 5 cars take?” it often fails completely.

Humans are capable of both understanding language and logical thought, I’m not sure if the latter will ever be easy for the LLMs to do, and perhaps older Symbolic approaches to AI might perform better in this space.