Prof. Emily M. Bender(she/her)Dec 24, 2022

Super frustrated with all the cheerleading over chatbots for search, so here's a thread of presentations of my work with Chirag Shah on why this is a bad idea. Follow threaded replies for:

op-ed
media coverage
original paper
conference presentation

Please boost whichever (if any) speak to you.

Show threadProf. Emily M. Bender(she/her)Dec 24, 2022

Chatbots are not a good replacement for search engines

https://iai.tv/articles/all-knowing-machines-are-a-fantasy-auid-2334

All-knowing machines are a fantasy | Emily M. Bender and Chriag Shah

The idea of an all-knowing computer program comes from science fiction and should stay there. Despite the seductive fluency of ChatGPT and other language models, they remain unsuitable as sources of knowledge. We must fight against the instinct to trust a human-sounding machine, argue Emily M. Bender & Chirag Shah.

IAI TV - Changing how the world thinks
Show threadProf. Emily M. Bender(she/her)Dec 24, 2022

Chatbots are not a good UI design for information access needs

https://www.technologyreview.com/2022/03/29/1048439/chatbots-replace-search-engine-terrible-idea/

Chatbots could one day replace search engines. Here’s why that’s a terrible idea.

Language models are mindless mimics that do not understand what they are saying—so why do we pretend they’re experts?

MIT Technology Review
Show threadProf. Emily M. Bender(she/her)Dec 24, 2022

Chatbots-as-search is an idea based on optimizing for convenience. But convenience is often at odds with what we need to be doing as we access and assess information.

https://www.washington.edu/news/2022/03/14/qa-preserving-context-and-user-intent-in-the-future-of-web-search/

Q&A: Preserving context and user intent in the future of web search

In a new perspective paper, University of Washington professors Emily M. Bender and Chirag Shah respond to proposals that reimagine web search as an application for large language model-driven...

UW News
Show threadProf. Emily M. Bender(she/her)Dec 24, 2022

Chatbots/large language models for search was a bad idea when Google proposed it and is still a bad idea even when coming from Meta, OpenAI or You.com

https://dl.acm.org/doi/10.1145/3498366.3505816

Situating Search | Proceedings of the 2022 Conference on Human Information Interaction and Retrieval

ACM Conferences
Show threadProf. Emily M. Bender(she/her)Dec 24, 2022

Language models/automated BS generators only have information about word distributions. If they happen to create sentences that make sense it's because we make sense of them. But dis-connected "information" inhibits the broader project of sense-making.

https://www.youtube.com/watch?v=VY1GHbU_FYs&list=PLn0nrSd4xjjY3E1qxXpWDoF7q-Q3d6g_A&index=17

Situating Search

YouTube
Show threadProf. Emily M. Bender(she/her)Dec 24, 2022

We must not mistake a convenient plot device — a means to ensure that characters always have the information the writer needs them to have — for a roadmap to how technology could and should be created in the real world.

https://mindmatters.ai/2022/12/why-we-should-not-trust-chatbots-as-sources-of-information/

Why We Should Not Trust Chatbots As Sources of Information

On a deeper note, they say, the pursuit of absolutely certain Correct Information suffers from a fundamental flaw — it doesn’t exist.

Mind Matters
Show thread[email protected] movedDec 25, 2022
@emilymbender thanks for sharing your insight. An anecdote from my toying around with #chatGPT: I asked it to show me an example of a program written in an imaginary combination of the best from the programming languages #Python, #Julia, #golang, and #Rust.
It wrote me a nice piece of pseudo-code that made sense. Furthermore, it could explain to me which traits represented characteristics from each language. Although it's probably not, it gave me an impression of creativity
Show threadDag Ågren ↙︎↙︎↙︎Dec 25, 2022

@arildsen @emilymbender Well the thing with chatbots like ChatGPT is that they are very good at exactly that: giving you an IMPRESSION that they are good at something.

But they will absolutely lie through their teeth to do it, and it will be believable lies.

Show threadCarola Schubert
@WAHa_06x36 @emilymbender @arildsen The problem begins with the linguistically fuzzy insinuation of lying. A Chatbot can only produce results in response to a prompt and cannot lie because reflection or morality or any intention as a consciously acting agent is missing. The result can seem like a lie to us because nonsense is possibly presented like facts.
Dec 26, 2022 at 2:13amWeb
Show threadDag Ågren ↙︎↙︎↙︎Dec 26, 2022
@cognisize @emilymbender @arildsen Not the point being discussed, though, is it? The question isn’t if it is moral for an AI to lie. It is that an AI will act in a manner indistinguishable from a human lying, which means it is less than useless, and actively harmful.
Show threadCarola SchubertDec 26, 2022
@WAHa_06x36 @emilymbender @arildsen Yes, I totally agree.