it's B! Cavello 🐝

This is some cool research, but I also fundamentally think we should be encouraging people NOT to use #TextGenerators as search.

https://www.linkedin.com/feed/update/urn:li:activity:7100577024452300800/

#TagZone #LLM #AI #MachineLearning #TechPolicy

Dave Guarino on LinkedIn: Sharing an example of some research I'm doing comparing how different…

Sharing an example of some research I'm doing comparing how different large language models (LLMs / AI) perform on complicated government program questions…

Aug 25, 2023 at 12:08amWeb
Show threadit's B! Cavello 🐝Aug 25, 2023

Text generators generate text.

They were not designed to search, and until they are built to do that, I think it’s super irresponsible to act otherwise.

I think that calling them "text generators" instead of "LLMs" or "GPTs" can help to clarify what they were designed to do.

Show threadit's B! Cavello 🐝Aug 25, 2023

I can't say for sure, but I actually think that if we called these things "text generators," we wouldn't even need a word for the misleadingly named "hallucinations" of these systems.

These tools weren't designed to be be truthful or factual, of course. They were designed to generate text.

A text generator generating text isn't a hallucination, it's just doing the thing it was built to do.

Show threadit's B! Cavello 🐝Aug 25, 2023

I am honestly excited about what AI tools could enable. Even just narrowing to text generators, I think there's going to be a ton of awesome creative stuff people figure out how to do leveraging these tools.

But they are tools.

And they were made by people to do a particular thing. That thing can be really impressive!

But it isn't search.

https://mastodon.publicinterest.town/@b_cavello/110195030804262597

it's B! Cavello 🐝 (@[email protected])

Attached: 1 image How I’ve been feeling about all these #GPT for search updates 🥴

Hometown
Show threadit's B! Cavello 🐝Aug 25, 2023
Thinking of using the hashtag #TalkBetterAboutAI for this kind of thing...
Show threadit's B! Cavello 🐝Aug 25, 2023
Oops! Broke the thread, but there's more here, if you're so inclined: https://mastodon.publicinterest.town/@b_cavello/110947429317025603
it's B! Cavello 🐝 (@[email protected])

Another way to #TalkBetterAboutAI is to try to avoid personifying language. This is TOUGH. Personifying stuff is so useful for explaining things. It's really hard to avoid, and often explaining things more accurately can take a couple more words. It is tough. But I think it's worth it, too. By talking about these tools as tools, it helps us recognize both our own agency and impact as users of the tools, but it also can help clarify the role of the people who created these tools.

Hometown
Show threadStudio 8502 Aug 26, 2023
@b_cavello @shyra I have had good results using ChatGPT for generating longer form TTRPG character biographies using a few sentences of concept. For example, I had it generate a Lawful Good necromancer who sells mummies as domestic servants.
Show threadit's B! Cavello 🐝Aug 27, 2023
@shyra @mos_8502 that sounds like a pretty appropriate use of these tools! They can be helpful for first drafts or using to modify and evolve text to spur creativity
Show threadStudio 8502 Aug 27, 2023
@b_cavello @shyra Yeah, if the task at hand doesn’t rely on real world facts, then “hallucinations” don’t matter.
Show threadMaxi 12x 💉Aug 26, 2023
@tante ^^
Show threadAiluridaeAug 26, 2023
@b_cavello how much text could a text generator generate if a text generator could generate text?
Show threadLaux Myth (aka Martin)Aug 27, 2023
@b_cavello
Loren ipsum just not in fake Latin.
Show threadsbiAug 31, 2023
@b_cavello "Sophisticated parrots" I call them.
Show threadPavel A. SamsonovOct 10, 2023
@b_cavello Love this. I was just work on a piece about how "generative AI" fits into the tradition of Lorem Ipsum - content that looks realistic enough for its purpose without being "real"
Show threadBryce WilleyAug 28, 2023
@b_cavello Simon Willison calls them word calculators, which I like a lot because it doesn't have any sort of jargon. Text generators is a bit more accurate though! https://simonwillison.net/2023/Apr/2/calculator-for-words/
Think of language models like ChatGPT as a “calculator for words”

One of the most pervasive mistakes I see people using with large language model tools like ChatGPT is trying to use them as a search engine. As with other LLM …

Show threadit's B! Cavello 🐝Aug 28, 2023
@brycew yes, I’ve seen this! I don’t really love calculators because I think it feels overly precise, when in fact part of what makes these systems so compelling compared to prior approaches is the amount of randomness/noise in the generation process (also they can do plenty of non-word things). I do think that it’s definitely the same vein.
@emilymbender has also written at length about these tools and has some apt metaphors for thinking about how they work.
Show threadProf. Emily M. Bender(she/her)Aug 28, 2023

@brycew

Thanks, @b_cavello ! Have you see this paper, specifically on why LLMs are not a match for search?

https://dl.acm.org/doi/10.1145/3498366.3505816

Related op-ed:

https://iai.tv/articles/all-knowing-machines-are-a-fantasy-auid-2334

Situating Search | Proceedings of the 2022 Conference on Human Information Interaction and Retrieval

ACM Conferences
Show threadit's B! Cavello 🐝Aug 28, 2023
@emilymbender @brycew I had not! Thanks so much for sharing
Show threadMichael BuschAug 25, 2023

@b_cavello Checking; every single one of those generated responses appears to be wrong: https://www.dhs.state.il.us/page.aspx?item=15070

Or at least outdated; depending on if a given generator is simply duplicating old scraped text verbatim or not.

Show threadit's B! Cavello 🐝Aug 25, 2023
@michael_w_busch Arguably, even if it is... that's wrong.
Show threadThunderFist!!!Aug 26, 2023
@b_cavello @pluralistic has taken to calling them “Plausibe Sentence Generators”.