Mastodawn

Car Wash Test on 53 leading AI models: "I want to wash my car. The car wash is 50 meters away. Should I walk or drive?"

https://lemmy.world/post/43503268

Car Wash Test on 53 leading AI models: "I want to wash my car. The car wash is 50 meters away. Should I walk or drive?" - Lemmy.World

Screenshot of this question was making the rounds last week. But this article covers testing against all the well-known models out there. Also includes outtakes on the ‘reasoning’ models.

Show thread

turboSnail Feb 25

Well, they are language models after all. They have data on language, not real life. When you go beyond language as a training data, you can expect better results. In the meantime, these kinds of problems aren’t going anywhere.

Show thread

VoterFrog

Why act like this is an intractable problem? Several of the models succeeded 100% of the time. That is the problem “going somewhere.” There’s clearly a difference in the ability to handle these problems in a SOTA models compared to others.