RE: https://mastodon.social/@MikeElgan/116602163994915587
It sounds from the very small amount of information in the article like the problem was that the system was reading the names in the wrong order. That’s most likely a human organization / data wrangling error, not gen AI slop.
If so — big if! speculating!! — this is a textbook example of someone instinctively using AI as an accountability sink. 🧵
In the coming years, I expect to hear a whole lot of “Don’t blame me!! Gen AI did it! The AI dog ate my homework!!”
One of my biggest wishes right now is that we get crystal clarity, both legally and culturally, that if a computer system screws up (AI or otherwise) then that’s the responsibility of the people who •chose to use• the system.
Note that I say “chose to use,” not just “built.” I don’t mean to slough off responsibility as a person who does build computer systems, but “ it’s our vendor’s fault” or “ the tech intern did it” are also common ways of evading responsibility.
You chose to use it? You chose to trust the builder? You need to take responsibility for that. If the vendor / engineers / implementors deceived you, you need to show that you were actively deceived — and aren’t just trying evade responsibility for your own decisions when you’re supposed to be the decision-maker.
@inthehands don't worry, when the stakes get higher, the humans overseeing them will become the accountability sinks, as LLMs aren't persons...
"The market’s bet on AI is that an AI salesman will visit the CEO of Kaiser and make this pitch: “Look, you fire 9/10s of your radiologists, saving $20m/year, you give us $10m/year, and you net $10m/year, and the remaining radiologists’ job will be to oversee the diagnoses the AI makes at superhuman speed, and somehow remain vigilant as they do so, despite the fact that the AI is usually right, except when it’s catastrophically wrong.
“And if the AI misses a tumor, this will be the human radiologist’s fault, because they are the ‘human in the loop.’ It’s their signature on the diagnosis.”
This is a reverse centaur, and it’s a specific kind of reverse-centaur: it’s what Dan Davies calles an “accountability sink.” The radiologist’s job isn’t really to oversee the AI’s work, it’s to take the blame for the AI’s mistakes."
—https://doctorow.medium.com/https-pluralistic-net-2025-12-05-pop-that-bubble-u-washington-8b6b75abc28e
Paul Cantrell
punIssuer