Anders EskildsenOct 22, 2025
Kevin Marks

Key findings:
45% of all AI answers had at least one significant issue.
31% of responses showed serious sourcing problems – missing, misleading, or incorrect attributions.
20% contained major accuracy issues, including hallucinated details and outdated information.
Gemini performed worst with significant issues in 76% of responses, more than double the other assistants, largely due to its poor sourcing performance.

https://www.bbc.co.uk/mediacentre/2025/new-ebu-research-ai-assistants-news-content

Largest study of its kind shows AI assistants misrepresent news content 45% of the time – regardless of language or territory

An intensive international study was coordinated by the European Broadcasting Union (EBU) and led by the BBC

Oct 22, 2025 at 11:51amWeb
Show threadKonosocioOct 22, 2025
@KevinMarks I think we can improve on that by creating systems for DIY news - correct only 20% of the time but that is a loss we are willing to bear for the incredible advantage of only reading what you actually want to read.
Show threadWorikOct 23, 2025
@konosocio @KevinMarks Crikey! That is the misinformation problem in a nutshell!
Show threadifraudingOct 22, 2025
@KevinMarks
We are so 🤏🏼 close to AGI. I promise!!1!
Show threadDagOct 22, 2025
@KevinMarks The amount of capital spent on this is humongous, and the value is not correlating. This capital could be a lot more wisely be spent on solving the #ClimateCrisis or education, health aid or hunger in developing countries.
Show threadSassinake! ᑐ ∪ ∩ ⊂Oct 22, 2025

@KevinMarks

'mansplaining as a service'

Show threadJerryJhirdOct 22, 2025
@KevinMarks crazy. thats almost half the percentage of how much BBC misrepresents news content
Show threadKanaMaunaOct 22, 2025

@KevinMarks

So slightly better than a coin toss. Good job everybody, let’s burn a few more forests to celebrate.

Show threadteledyn 𓂀Oct 22, 2025

@KevinMarks

Something something silk purse something something sow's ear …

😅

Show threadTristan Nitot✓Oct 22, 2025
@KevinMarks good stuff, thanks for sharing, happy to have news from you again, sorry I did not follow you earlier :-) (I wish I did...)
Show threadToni AittoniemiOct 22, 2025
@KevinMarks This does not surprise me one bit.
The network only ”knows” within it’s training data.
The rest is the reader’s gullibility not knowing better.
Show threadayOct 22, 2025
@KevinMarks If it's failing so bad at just processing text, imagine how inaccurate it must be at complex tasks. I can hear popping sounds.
Show threadWorikOct 23, 2025

@ayameow @KevinMarks I could answer that question, if you care.

I have a lot of experience, given the short time, testing that

Show thread3rdiOct 22, 2025
@KevinMarks This is the purpose of AI
Show threadAlexis Mostoller ArtOct 23, 2025
@KevinMarks I am quite confident that a few years down the road, we are going to see some startling positive correlations between AI usage and early-onset dimentia. Use it or lose it is the anthem of neural pathways, and if you're not using them... well...
Show threadlaydrosOct 23, 2025
@KevinMarks geez, what did they use to train these things, the internet!!?!
Show threadmistermupsOct 23, 2025
@KevinMarks 55% of the time, it works every time. All that cash and resource for something that's objectively worse than 'Sex Panther - By Odeon'. I mean you couldn't make it up really.
Show threadEd SandersOct 23, 2025
@KevinMarks I’m starting to think of AI as a guy boiling water on a stove, but convincing suckers (and there are plenty who call themselves investors and bankers) that they are in the early days of developing fusion.
Show threadPiranhallamaOct 24, 2025

@KevinMarks whilst I'm glad a proper study has been done, I'm not really sure if it matters. When such a high percentage of the public believe something, the truth becomes largely irrelevant. Saying AI isn't great seems like preaching to the converted now.

I view AI like I view cults. I'm pretty sure that deep down, at home, with the curtains drawn and the lights off when cult followers would admit that some things they tow the line on are not correct. They also don't care as they'd rather be in than out. For whatever reason (I guess media and hope) a lot of people I know who support and use AI seem to gloss over all the negatives with "yes, well...", "is still new...", "it's improving..." and seem very happy with mediocre results it yields them. If it was any other thing I'm pretty sure they wouldn't touch it with a barge pole. It's easier to type an AI prompt than read a book that's mostly wrong. Maybe it's an effort thing.