Simon WillisonFeb 9, 2023

The biggest question for me about large language model interfaces - ChatGPT, the new Bing, Google's Bard - is this:

How long does it take for regular users (as opposed to experts, or people who just try them once or twice) to convince themselves that these tools frequently makes things up that aren't accurate?

And assuming they figure this out, how does knowing it affect the way they use these tools?

Show threadmikesten
@simon To be fair, it's taken _me_ a while to properly understand it, despite thinking I had a good handle on it. And - ridiculously - it was ChatGPT confidently reporting a completely made up tally of Scrabble scores that drove the point home. I expected it to get obscure stuff wrong but adding up a bunch of numbers?
Feb 9, 2023 at 4:12pmWeb
Show threadSimon WillisonFeb 9, 2023
@mikesten yeah, the "wait a second, this thing is a COMPUTER and it can't even do MATH?" learning moment is a pretty powerful one!
Show threadmikestenFeb 9, 2023
@simon On the bright side, it gave us both 200 instead of giving Syl 240 and me 220. So.. I sort of owe it a beer.