Chilly  đź›ˇď¸Ź  11h ago
Kees van der Leun
Google's AI overviews give wrong answers 9% of the time (1.25 billion times a day), and in over half of the cases point to websites that don't actually support the answer given.
https://futurism.com/artificial-intelligence/google-ai-overviews-misinformation
Analysis Finds That Google’s AI Overviews Are Providing Misinformation at a Scale Possibly Unprecedented in the History of Human Civilization

A new analysis commissioned by The New York Times suggests that Google's AI Overviews are wrong an astonishing percentage of the time.

Futurism
4:58amWeb
Show threadStarkRG12h ago
@Sustainable2050 Having that high a wrong answer rate makes using them for answers absolutely useless. To know whether your answer is one of the barely more than 9-in-10 right answers, you have to either already know the answer, which you don't because you're looking for the answer, or you'll need to go read actual sources to find out, which is what you should have done in the first place. LLMs are terrible at just about everything.
Show threadalsvha12h ago
@StarkRG @Sustainable2050 exactly - it's one of my main arguments at work when ever they "encourage" us to use LLMs to gain the massive "performance benefits" in solving/development.
Show threadStarkRG12h ago
@alsvha @Sustainable2050 There's really only one halfway decent use for LLMs and that's as an accessibility aid for when developers and designers don't do their damned jobs properly and just *include* accessibility features instead of relying on an energy-inefficient, ethically-questionable technology to bridge the gap.
Show threadalsvha11h ago

@StarkRG @Sustainable2050 I can find use cases and I can see benefits in some areas - but whether it's worth the cost (environmental, ethical appropriation, vendor lock-in, skill deterioration etc) ; I'm very much not sure off.

I become even more of an anti-person, because it's also being forced upon everything/everybody with dubious claims in a multitude of non-working scenarios.
And when managers then claim "85% performance gain" and I hear my colleagues talk about 'Claude' as a person and more about how to prompt than how to solve, I get so very frustrated.

Show threadStarkRG11h ago
@alsvha I'm not saying there aren't any other use cases, I'm saying that I can't really think of one use case that isn't already handled at least as well by a more environmentally-friendly and ethically-sound technology. The only reason accessibility kind of qualifies as a good use case is because, although accessibility *should* be a standard feature for everything, it's usually a second thought, if that.
Show threadMartin Vermeer FCD11h ago
@StarkRG @alsvha One valid use case I have found is machine translation between two languages that you are fluent in. You still have to do the manual check, but it may save you time and effort, and may even prevent the typical mistakes that writers non-native in the target language make.
Show threadMartin Vermeer FCD11h ago

@Sustainable2050

> in over half of the cases point to websites that don't actually support the answer given.

Actually the old practice of simply making up non-existent web sites was preferable: at least you didn't have to read anything with comprehension

Show threadsam 10h ago
I tried duck.ai a few times and most of the source links it gave me were either 404 or unrelated.
Show threadThomas Cloer10h ago

@Sustainable2050 Here is the original piece, not so much black-and-white:

https://www.nytimes.com/2026/04/07/technology/google-ai-overviews-accuracy.html?unlocked_article_code=1.ZlA.EO4j.qmTIDhIBGKRh&smid=url-share

You are welcome.

How Accurate Are Google’s A.I. Overviews?

The company’s A.I.-generated answers look authoritative, but they draw on an array of sources, from trustworthy sites to Facebook posts.

The New York Times
Show threadRay McCarthy9h ago
@Sustainable2050
There was study that showed mostly AI LLMs "shorten" rather than summarise and will likely miss a summary already in the document. They have zero context for what's important. No intelligence, so this is no surprise.