Ars Technica3d ago
Analysis finds Google AI Overviews is wrong 10 percent of the time
Is 90 percent accuracy good enough for a search robot?
https://arstechnica.com/google/2026/04/analysis-finds-google-ai-overviews-is-wrong-10-percent-of-the-time/?utm_brand=arstechnica&utm_social-type=owned&utm_source=mastodon&utm_medium=social
Show threadJoe ❌👑
@arstechnica It only does that well because most questions it is asked are very easy to answer. For the nontrivial questions it does a good deal worse, and it can't distinguish satire from reality, a feature that people are exploiting (the new version of Googlebombing).
Apr 7 at 5:10pmWeb
Show threadDavid Nash3d ago

@not2b @arstechnica

That, and “90 percent accuracy” is often not the same as “90 percent utility”.

If you have a bread recipe with 10 steps, and an “AI” summary inadvertently deletes 1 of them, the recipe is “90%” correct, but if the deleted step is “add yeast”, the bread ranges from unpalatable to inedible, and if the deleted step is “add flour,” it completely fails to be bread.