Paul Cantrell

RE: https://mastodon.social/@MikeElgan/116602163994915587

It sounds from the very small amount of information in the article like the problem was that the system was reading the names in the wrong order. That’s most likely a human organization / data wrangling error, not gen AI slop.

If so — big if! speculating!! — this is a textbook example of someone instinctively using AI as an accountability sink. 🧵

May 19 at 4:48pmWeb
Show threadPaul CantrellMay 19

In the coming years, I expect to hear a whole lot of “Don’t blame me!! Gen AI did it! The AI dog ate my homework!!”

One of my biggest wishes right now is that we get crystal clarity, both legally and culturally, that if a computer system screws up (AI or otherwise) then that’s the responsibility of the people who •chose to use• the system.

Show threadPaul CantrellMay 19

Note that I say “chose to use,” not just “built.” I don’t mean to slough off responsibility as a person who does build computer systems, but “ it’s our vendor’s fault” or “ the tech intern did it” are also common ways of evading responsibility.

You chose to use it? You chose to trust the builder? You need to take responsibility for that. If the vendor / engineers / implementors deceived you, you need to show that you were actively deceived — and aren’t just trying evade responsibility for your own decisions when you’re supposed to be the decision-maker.

Show threadDeterioratedStuccoMay 19
@inthehands
Well, that's the end of vehicle-driver automation. But you're right.
Show threadPaul CantrellMay 19
@SoftwareTheron
…I mean, yes. Unless you have a system that really, truly does a better job than a human. And that seems like the right liability regime: you’re responsible for causing a crash regardless, so what setup will make you less likely to cause one?
Show threadDeterioratedStuccoMay 19
@inthehands
I'm pretty clear that *under the right circumstances* (i.e. with the right people doing the development) it will be possible to get a statistically-safer automated vehicle. But nobody will ever buy it. Shame.
Show threadpunIssuerMay 19

@inthehands don't worry, when the stakes get higher, the humans overseeing them will become the accountability sinks, as LLMs aren't persons...

"The market’s bet on AI is that an AI salesman will visit the CEO of Kaiser and make this pitch: “Look, you fire 9/10s of your radiologists, saving $20m/year, you give us $10m/year, and you net $10m/year, and the remaining radiologists’ job will be to oversee the diagnoses the AI makes at superhuman speed, and somehow remain vigilant as they do so, despite the fact that the AI is usually right, except when it’s catastrophically wrong.

“And if the AI misses a tumor, this will be the human radiologist’s fault, because they are the ‘human in the loop.’ It’s their signature on the diagnosis.”

This is a reverse centaur, and it’s a specific kind of reverse-centaur: it’s what Dan Davies calles an “accountability sink.” The radiologist’s job isn’t really to oversee the AI’s work, it’s to take the blame for the AI’s mistakes."
—https://doctorow.medium.com/https-pluralistic-net-2025-12-05-pop-that-bubble-u-washington-8b6b75abc28e

The Reverse-Centaur’s Guide to Criticizing AI

My speech for U Washington’s Neuroscience, AI and Society lecture series.

Medium
Show threadPaul CantrellMay 19
@punissuer
Yeah, I was posting that same link just the other day
Show thread😀🚲May 19
@inthehands this is going to reveal that most decision makers lack the technical knowledge to make most decisions without a trustworthy advisor. Also, I doubt they can be held accountable by people who don't understand the decisions and/or implications.
Show threadUnassumingKiwiMay 19

@inthehands
I've been told at 2 separate companies I've worked at that we were purchasing external systems instead of building them ourselves explicitly so that blame can be shifted if something goes wrong.

So I always assume its a *deliberate* strategy until proven otherwise

Show threadFish Id Wardrobe ⁂May 19

@inthehands
> You chose to use it? You chose to trust the builder? You need to take responsibility for that.

Well, yes, but also, as someone pointed out here recently: "All Software Is Malware Now. Every auto updater is a Trojan horse."

sometimes it's not about trusting the builder, but about choosing between weevils and hoping you've picked the lesser.

Show threadTravis BriggsMay 19
@inthehands Absolutely. People at work are suggesting that when you copy from a Gemini/ChatGPT/etc reply, it should automatically prepend "Well Gemini says: ..."
Show threadTravis BriggsMay 19
@inthehands See also: https://www.nytimes.com/2026/05/19/business/media/future-of-truth-ai-quotes.html
‘The Future of Truth’ Contains Quotes Made Up by A.I.

Steven Rosenbaum, author of “The Future of Truth,” said he had started his own investigation after The New York Times asked about the fake quotes.

The New York Times
Show threadPaul CantrellMay 19

@audiodude

Yes and yes!

Show threadwbftwMay 19
@inthehands fwiw (haven’t verified):
Show threadwbftwMay 19
@inthehands a link to the post: https://www.reddit.com/r/antiwork/comments/1onhoxt/my_companys_new_ai_payroll_bot_decided_i_dont/
Show threadSteve HerseyMay 19

@wbftw @inthehands
Illegitimately withholding your pay is illegal. Tell HR to cut you a manual check or it's lawyers-and-state-AG time.

Then start your job search; these bozos are going down for the count no matter what the outcome of this particular issue.

Show threadwbftwMay 19
@n1xnx @inthehands Not sure for whom this unsolicited advice is, may be respond in the reddit thread?
Show threadSteve HerseyMay 19
@wbftw @inthehands
A good point. I hadn't realized it was a cross-platform post.
Show threadMichael J. CoffeyMay 19
@wbftw @inthehands As a former "business assistance officer" for a program of the US Small Business Administration (but not a lawyer), this may well be illegal. It might be worth checking with a lawyer.
Show threadTalya (she/her) 🏳️‍⚧️✡️May 19
@wbftw @inthehands don't know which country they are, but in most countries if your paycheck get delayed you can sue, often without proving damages. so i very much hope that it's the case there too.
Show threadÖlbaumMay 21
@Yuvalne @wbftw @inthehands They’re probably in that country without any regulations that protect individuals, where you can be terminated without pay for giving your notice.
Show threadMaddieM4May 19
@wbftw @inthehands the biggest category of theft has been wage theft for a long time. And why not, if nobody's punishing it?
Show threadShadSterlingMay 19
@wbftw @inthehands we already needed a civil analog to the public defender, to enforce the “laws” that only say you have a right to sue (if you can afford lawyers and years), looks like that’s starting to get a lot worse
Show threadSimon HearsMay 19
@inthehands sadly there has been no accountability prior to AI, so I’m not holding my breath. ToS always means no accountability … because software development is not a profession.
Show threadcratermoonMay 19
@inthehands Joseph Weizenbaum summed it up correctly in 1976, "there are certain tasks which computers *ought* not be made to do, independent of whether computers can be made to do them". For him, "since we do not now have any ways of making computers wise, we ought not now to give computers tasks that demand wisdom."
Show threadAlex FürstenauMay 19
@inthehands totally agree. Same with any other outsourcing like AWS or so. I didn't choose AWS or the AI, I chose you. So I am YOUR customer and you are responsible for your decisions. Sorry for the rambling.
Show threadarceuthobiumMay 19
@inthehands it’d also be nice to pin down the AI vendors as to whether their product is warranted as suitable for a specific task. Oh, you sold bookkeeping software that’s not actually suitable for bookkeeping? You lied about your software being suitable for educational use? You bought software that you knew was unsuitable for medical record keeping?
Show threadGreg BellMay 19
@inthehands because it goes in both directions. Low effort users just chuckle Woops the AI sucks lol and the AI promulgators explicitly disclaim responsibility while at the same time promising revolutionary wins, which cannot both be true at once
Show threadtian2992May 19
@inthehands boo-u(rns)
Show threadWulfy—Speaker to the machinesMay 19

@inthehands

I'm old enough to remember PC introduction to the office.

Folks were blaming "the computer" for their screw-ups.

This is why today, I say "AI is a learned skill"
Virtually every case of "AI broke shit" I examined is operator error.

Show threadPaul CantrellMay 19

@n_dimension

I mean, LLMs do frequently answer questions incorrectly, generate nonsensical data, and generally do things we would consider to be screw-ups. And (1) expecting this to happen, (2) having and executing a good plan to detect it / verify it / correct it / contain the failure / remove the failure from critical chains, and (3) knowing when simply not to use an LLM are all baseline managerial requirements for letting them into your org in the first place.

So if that’s what you mean by it being a learned skill, I’m with you: not that operator skill can make it be perfect, but rather than using the tool requires knowing what the tool is and is not good for.

Show threadWulfy—Speaker to the machinesMay 19

@inthehands

No tool is perfect.
Many can mail or kill.

AI is a nascent tech.
Steam engine boilers used to explode, detached fly wheels used to go for a walk through neighbourhood and machines ingest limbs and people to this day.

Unlike most folks, I see replacement of intellectual Labor by machines, a continuation of replacement of physical Labor.
The capitalists are nearing their goal of near total replacement of the proletariat.

Workers must unite against the capital.
#regulateAi Sovereign Models, public controls on AI.#antichrist

There is a reason Peter Thiel calls folk who want to regulate #Ai literal #antichrist

Show threadPaul CantrellMay 19
@n_dimension
…and my list of 3 items applies just the same to industrial machinery
Show threadcratermoonMay 19
@inthehands every organization I’ve worked with has issues with garbage data. Rarely do they take data consistency and cleanliness seriously. If they don’t have AI to blame they blame the db, or the etl system, or anything other than their own lazy, penny-pinching sloppiness.
Show threadPaul CantrellMay 19
@cratermoon
Correct. The advantage of gen AI is that it can more fully automate the blame: if it does •everything•, then you can blame it for everything.
Show threadAAA Trans PirateMay 19

@inthehands

which is its #2 main use case. the #1 use case is grifting everyone under the sun.

#AIisalibi