@alisynthesis @davidaugust fair enough. I changed up the problem completely and added some reasoning and it did pretty well. It appears to be generating code to solve the math. The only thing it missed is that very unripe bananas are green, not yellow.

James picks 40 apples on Monday. Then he picks 35 lemons on Tuesday. On Wednesday, he picks half as many bananas as he did apples, but five of them were very unripe. How many yellow fruits does James have?

@audioflyer79 @alisynthesis @davidaugust
The correct mathematical response to your question is either a statement of uncertainty (I can't answer that because I don't know what color the apples are or how ripe the lemons and non-unripe bananas are) or a request for clarification (what kind of apples? are the lemons ripe? how ripe are the ripe bananas?).

The fact that it provides a guess indicates that it has correctly understood what *you want it to say*.

It's not doing math. It's playing "what does the user expect?"

@audioflyer79 @alisynthesis @davidaugust
I don't want something that's sold as "useful because it is knowledgeable and capable enough to perform jobs for me" to operate at a grade school level of understanding. I want it to grasp nuance and call out uncertainty, so that it will do precisely what I need it to do.

The problem with this challenge, as posed to a computer, is that it is not fundamentally a math or language understanding problem.

It is an *inference* problem. It is assessing the model's ability to infer properties from insufficient information (what color are apples, ripe bananas, unripe bananas, and lemons?"). (Questions like these are one reason standardized tests produce biased data, because people exposed to different environments reach different answers.)

But the Apple paper is posing a *math* problem that enables them to probe the model's *language understanding*.

The success metric for the math problem is "did the model demonstrate understanding of the text when not specifically trained to do so?" (and therefore loses its utility once it is available publicly and can be included in their training data).

The success metric for the inference problem is "can the model correctly match fruit names to a color label?". This metric doesn't illustrate understanding, because it is testing precisely the kind of linguistic pattern-matching the models are designed to do.

@audioflyer79 @davidaugust @alisynthesis This is actually very important.

LLMs do not "forget" the way humans do. They don't have memory lapses.

Humans have memory lapses and difficulty recalling facts they know.

But the nature of computers is to remember perfectly. LLMs exist because they remember perfectly and look for patterns in that memory.

If it is to be useful at interpreting human commands and understanding human expectations (see "now anyone can code" PR), it needs to be encoding concepts, not characters.

People are making big claims that these machines are somehow conscious or intelligent and are able to understand abstract concepts.

If an inference machine with perfect memory states unequivocally at one point that unripe bananas are yellow, and then later states unequivocally that unripe bananas are green, then it is not storing and retrieving conceptual information.

Any claim of generalization cannot involve this kind of concept.

In the battle between Cog and Cyc, LLMs are Cyc writ large.

Cog: https://en.wikipedia.org/wiki/Cog_(project)
Cyc: https://en.wikipedia.org/wiki/Cyc