Prof. Emily M. Bender(she/her)Jan 11

Health experts: Your synthetic text "AI" overviews are misleading, for example see this about liver function tests.

Google: Okay, we'll block "AI" overviews on that query.

The product is fundamentally flawed and cannot be "fixed" by patching query by query.

A short đŸ§”>>

https://www.theguardian.com/technology/2026/jan/11/google-ai-overviews-health-guardian-investigation

‘Dangerous and alarming’: Google removes some of its AI summaries after users’ health put at risk

Guardian investigation finds AI Overviews provided inaccurate and false information when queried over blood tests

The Guardian
Show threadProf. Emily M. Bender(she/her)Jan 11

"Block that search in particular" is also what Google did when the inherent racism in their image search algorithm was pointed out, in 2018.

https://www.theguardian.com/technology/2018/jan/12/google-racism-ban-gorilla-black-people

>>

Google's solution to accidental algorithmic racism: ban gorillas

Google’s ‘immediate action’ over AI labelling of black people as gorillas was simply to block the word, along with chimpanzee and monkey, reports suggest

The Guardian
Show threadProf. Emily M. Bender(she/her)Jan 11

And when Dr.
@safiyanoble pointed out the racist results returned for queries like "black girls" in ~2016. See her amazing book Algorithms of Oppression:

https://nyupress.org/9781479837243/algorithms-of-oppression/

>>

Algorithms of Oppression

A revealing look at how negative biases against women of color are embedded in search engine results and algorithms Run a Google search for “Black girls”...

NYU Press
Show threadProf. Emily M. Bender(she/her)Jan 11

Back to the current one, the quotes from Google in the Guardian piece are so disingenuous:

>>

Show threadProf. Emily M. Bender(she/her)Jan 11

I've been shouting about this since Google first floated the idea of LLMs as a replacement for search in 2021. Synthetic text is not a suitable tool for information access!

https://buttondown.com/maiht3k/archive/information-literacy-and-chatbots-as-search/

/fin (for now)

Information literacy and chatbots as search

By Emily This post started off as a thread I wrote and posted across social media on Sunday evening. I'm reproducing the thread (lightly edited) first and...

Mystery AI Hype Theater 3000: The Newsletter
Show threadJPJan 11
@emilymbender we're living through a mass psychological engineering campaign and the results have been, and will continue to be, horrifying https://azhdarchid.com/are-llms-useful
Are LLMs 'useful'?

Here's a thought experiment...

Azhdarchid
Show threadNicole ParsonsJan 12

@jplebreton @emilymbender

The question is also "LLM'S are useful to whom?"

The wealthiest seem overjoyed with it so far.

So much so, they are funding one of the largest coercive & forced user adoption campaign in history.

https://www.forbes.com/sites/mattdurot/2025/07/17/bill-gates-charles-koch-and-three-other-billionaires-are-giving-1-billion-to-enhance-economic-mobility-in-the-us/

It's the best at:
1. Election interference
2. Malign influence campaigns
3. Automated cyberwarfare
4. Manipulation of public sentiment
5. Automated hate campaigns
6. Plausible deniability for funding a fascist movement
7. Frying the planet

Five Billionaires Pledged $1 Billion To Boost Economic Mobility Using AI

Five of America’s top philanthropists are teaming up for a new venture aimed at helping low-income Americans rise from poverty. An AI giant has signed on to help.

Forbes
Show threadJPJan 12
@Npars01 @emilymbender yeah, it's clearly one of those technologies that accelerates just about everything patriarchal white supremacist capitalism does in various ways, and provides a greater means of plausible deniability to the people behind that than previous systems. it enables the monsters running the world to Capitalism Harder, at the exact moment when we need to be doing the opposite and take better care of one another and our planet. so in that sense it's definitely working as designed.
Show threadLRJan 12
@jplebreton @Npars01 @emilymbender or not designed, as this shit was ripped from labs before it was ready, because moneydudes got the fomo. the devil finds work in underengineered solutions.
Show threadJP
@lritter @Npars01 @emilymbender of course yeah, a lot of modern corporate "design" processes are like that, a thing is shown to have certain strategy-affine effects and it's immediately productized with no long term thinking needed.
Jan 12 at 3:32pmTusky