The biggest question for me about large language model interfaces - ChatGPT, the new Bing, Google's Bard - is this:
How long does it take for regular users (as opposed to experts, or people who just try them once or twice) to convince themselves that these tools frequently makes things up that aren't accurate?
And assuming they figure this out, how does knowing it affect the way they use these tools?
@codedread I have a good understanding of why they lie so much: all they're ever doing is predicting the next word in a sequence of words based on their training data
They have no concept of truth - they just know statistically which words are most likely to follow "The Kennedy assasination was a conspiracy by ..." - based on the TBs of scraper data that was used to build their models
The fact that they get anything right at all is pretty astonishing!
@simon @codedread I really wish they would remove the personification and chat UI. It should not look the same as a box where I send messages to people and people reply to me. User expectations would be better matched to the tools with a better explanation of the prompt and response.
I really get a lot out of using GPT-3 through the playground interface (thanks to your intro, Simon) but I haven't been using the chat interfaces because it feels like the wrong tool.
@Rob_Russell @codedread oh that's really interesting - I hadn't thought about how strongly the chat interface reinforces the science fiction "AI" aspect of it all
The playground interface never seemed to click for a lot of people
Simon Willison
Jeff Schiller
Rob Russell