At a conference in Helsinki last year, I had a discussion with Ron Chrisley about language models. One of the things he wagered they couldn't do was mental imagery, and he used an old set of cogsci probes on mental imagery that GPT-3 failed miserably at. One of them included mental rotation, like turning a letter in your head to see what other letter it forms. Well, here's ChatGPT's first shot at it:
[email protected]:
What letter do you get if you rotate the letter 'p' 180 degrees?
ChatGPT: If you rotate the letter 'p' 180 degrees, you will get the letter 'd'.
[email protected]:
How about if you flip the letter 'd' horizontally, what letter do you get?
If you flip the letter 'd' horizontally, you will get the letter 'b'. Flipping a letter horizontally is also known as reflecting the letter across a vertical axis. When you reflect a letter horizontally, the letter will appear to be reversed, with the left side becoming the right side and the right side becoming the left side.
[Note: I did not ask for a condescending explanation]