Facebook (sorry: Meta) AI: Check out our "AI" that lets you access all of humanity's knowledge.
Also Facebook AI: Be careful though, it just makes shit up.
This isn't even "they were so busy asking if they could" --- but rather they failed to spend 5 minutes asking if they could.
Using a large language model as a search engine was a bad idea when it was proposed by a search company. It's still a bad idea now that it's being proposed by a social media company. Fortunately, Chirag Shah and I already wrote the paper laying out all the ways in which this is a bad idea.
In the popular press/general public-facing Q&A about our paper:
https://www.technologyreview.com/2022/03/29/1048439/chatbots-replace-search-engine-terrible-idea/
And let's reflect for a moment on how they phrased their disclaimer, shall we? "Hallucinate" is a terrible word choice here, suggesting as it does that the language model has *experiences* and *perceives things*. (And on top of that, it's making light of a symptom of serious mental illness.)
Likewise "LLMs are often Confident". No, they're not. That would require subjective emotion.
I went digging in the paper to see if they cite #StochasticParrots or Bender & Koller 2020 or Shah & Bender 2022. That is, did they read about why this is misguided and just press ahead anyway? Apparently not.
They do cite Blodgett et al 2020 (fabulous paper!)
https://aclanthology.org/2020.acl-main.485/
But in the strangest possible way. Are they reflecting on the possible harms their technology might engender? No, of course not. They're striving for TRUTH! And thus worried about "bias".
omg I laughed so hard when I got to the "hallucinate" line
It's a terrible bit of anthropomorphization, but *damn* it's funny to see the CREATOR saying that
If I were helping them copyedit it I might have suggested the line "Language Models Can Pull Stuff Out Of Their Hats That Makes, Like, *Zero* Sense, So YMMV k thx bai"
@emilymbender Perhaps I'm misunderstanding something, but this feels a bit like a strawman. AI generated results will have issues with "relevance, usefulness, and trustworthiness" but no one's saying they won't. None of these products are being presented as infallible oracles.
Some people, when the products are more well polished and no longer have the GIANT DISCLAIMERS they currently have, will over-trust, but answers from people also have these problems, right?
@emilymbender Sorry, I thought you'd have noticed that I quoted from your paper in my response. I have more than an academic interest here. My company is using AI models to do these things & I'm genuinely interested in good criticism.
Maybe it was the ethics angle that didn't work for me. Who else is writing good criticisms?
@emilymbender My first attempt to use it generated something that brilliantly mimicked a paper written by someone who had forgotten the assignment was due today and hadn’t read the reading material.
And as silly as that sounds, it highlights the core problem. It doesn’t know what it doesn’t know. So it will just plug garbage into the gaps that sounds real, but isn’t. And as a reader, you have no idea which parts are based on “facts” (aka unauthenticated or attributed stuff ingested somewhere unknown) and which are made up to sound right.
@emilymbender
That was a good reminder of how toxic my last job was. I was being encouraged to make decisions without enough info, for a group of people who had far more experience than I did. That ran very counter to my usual technique of asking lots of questions, finding the people with good ideas, and supporting them.
You just gave me a wake-up that I may have left there, but I’m still using that “I know what I’m talking about” style instead of couching questions and off-the-cuff theories as what they actually are. Ugh.
A gut punch that I needed. Thanks for taking the time to respond.
@emilymbender Amy Hoy proposed
> we should call it “artificial mansplaining,” always confident, rarely correct
@emilymbender
I'm somehow working "language models can hallucinate" into something! Anything!
Just have no idea of what yet.
This is a job for #UnderDweller (my subconscious). 😄
I haven't set up #HomeAssistant voice input at this point and have the hardware to do it.
@emilymbender Oh boy I can't wait for *this* to backfeed into itself like every other language model eventually does
Someone's going to try writing Wikipedia articles with this, then they get scraped back into v2 of the AI, and the loop closes in on itself.
Vedran Sekara
Prof. Emily M. Bender(she/her)
Leon Derczynski 🏔️✍🏻🌲☀️
Adam Collins
Aaron
Jed Brown
Clive Thompson
William Gunn
Jack Mott
Taylor Beauvais
Kee Hinckley
Dr. Dan Killam
David Jayatillake
Conor O'Neill
ginevra
Anthony (he/him)
Maarten
kaybeeque 🍁💪💪🍁
Gustavo Franco
Dark Sage Torunka 
Bernard Monsun
Mike Taylor 🦕
PC