I've seen a lot of awful and ridiculous AI hype in the past two years, but this weekend there was one that briefly took away my ability to even.

I recovered, and wrote up a newsletter post:

https://buttondown.email/maiht3k/archive/medical-science-shouldnt-platform-automating-end/

Medical Science Shouldn't Platform Automating End-of-Life Care

JAMA Network publishes a view from deep inside the hype By Emily Caduceus symbol by Eliot Lash In a viewpoint article published in JAMA Network last week,...

@emilymbender

If you thought social media posts after death were horrifying, this is what is awaiting you beforehand.

Thanks for keeping an eye on these things!

@emilymbender
I have a feeling that this would be no more accurate to someone's wishes than if decisions were made by just flipping a coin.

The point is not to determine the person's wishes, but to take the decision out of human hands so that no one has to feel responsible for it. Just flip a damn coin, then, and call it a decision made by Fate.

@artemis @emilymbender Absolutely. And: way less resources wasted. But many people crave for magic machines.
@emilymbender Thanks for this very important article Pr. Bender! I shared it on my linkedin, I worked during 4 years in public health here in Montreal (for University of Montreal), and I strongly believe that your opinion has to be heard. This is quiet scary...

@emilymbender

Problem: "most critically ill patients have never engaged in comprehensive advance care planning"

Solution: invade patients' privacy, make them even more leery of bringing up sensitive topics with their providers, entrust lots of sensitive (and marketable) data to a black box system that we already know will produce biased output, and use it for life-and-death decisions

But there's not much point debating it because "it's inevitable"🤦‍♀️

@emilymbender why not just prioritize getting patients to file their living wills/directives/authorizations instead of subjecting them to this appalling invasion of privacy and dehumanization? Because it’s not as sexy as AI that’s why.
@emilymbender yeah, let's have these decisions done by a glorified autocomplete function...

@emilymbender
The fact this was even proposed...

Welcome to The Future.

@emilymbender was this exact scenario a Twilight Zone, an Outer Limits or a Star Trek cautionary tale?

Let's go ahead and implement it anyway. We gotta do something with the trillion dollars of GPUs we bought.

@MrMozz @emilymbender
Probably all three. Maybe a Black Mirror episode too.
@emilymbender (Yakov Smirnoff voice) In AI dystopia, computer unplugs YOU

@emilymbender

"We published this Viewpoint because it is very interesting, somewhat scary, and probably inevitable."

https://jamanetwork.com/journals/jamainternalmedicine/article-abstract/2821367

#EndOfLife
#PalliativeCare
#AI

Artificial Intelligence as Surrogate Decision-Maker

When the Editors read this Viewpoint,1 we were startled by the overall idea that artificial intelligence could actually speak to decision-makers and clinicians concerning goals of care for a patient at the end of life who is unable to communicate. We found some of the assumptions for how this might...

@emilymbender "Why yes, if your husband Jim were able to speak, he would definitely want you to end his life as quickly as possible to free up this bed for paying customers. Have a great day!"
@emilymbender 👍 I believe medical science shouldn’t platform automatic care period.
The last thing I want in life – literally? – is a chatbot deciding whether I live or die. https://buttondown.email/maiht3k/archive/medical-science-shouldnt-platform-automating-end/
Medical Science Shouldn't Platform Automating End-of-Life Care

JAMA Network publishes a view from deep inside the hype By Emily Caduceus symbol by Eliot Lash In a viewpoint article published in JAMA Network last week,...

@emilymbender my response upon reading your description of the article.

@emilymbender I gasped out loud when reading the title. And then I saw that they imagined that based on full audio recordings of doctor-patient conversations and I noped the hell out of there.

In 2005 I decided what I wanted to do professionally was to use computers and maths to improve medicine, which was super niche back then. How far I was from imagining the nightmare into which my dream would turn.

@emilymbender Getting ended by an AI deciding for you because of your meme search history is such a dystopian way to go!
@emilymbender @catsalad As someone who had to make those very decisions for my husband, who was unresponsive and dying from a glioblastoma: I am glad I have never met these ghoulish idiots in person, because my children need me to stay out of prison.

@wendynather @catsalad

I'm so sorry for your loss. That experience must have been so difficult. (And still, "AI" would not have helped.)

@emilymbender @catsalad Anyone who thinks that searching through historical data can provide the answer to something like that, has never faced it.
@emilymbender another one for @ianb, and quite a doozy, to boot!

@emilymbender
Ouch. And:

"must be grounded in an* understanding not only of the actual"

@emilymbender
Well of course it's inevitable that we'll all keep outsourcing our thinking to MOLE (Machine Only Learning Emulators). If we didn't, we'd only let our emotions cloud our decisions about letting clouds think for us. In fact, I think it makes sense to outsource the decisions about when to outsource our decisions too.

What could go wrong? : P

https://strypey.dreamwidth.org/1120.html

Captcha Check

@emilymbender truly horrifying. I'd personally be more comfortable with a coin flip.
@emilymbender I fully agree that this is not a place for AI. At all. It is unethical and immoral to outsource the burden of making such momentous decisions to some souped-up statistics program. No matter how hard it is for loved ones to make these calls (yes, I’ve been there), it is something that one needs to do oneself. If there is no next of kin to decide, then doctors should decide on purely medical prognosis. Not on the patient’s Facebook history or such.
@emilymbender On the plus side, the AI could be informed by the Soylent factory whether there's a shortage of green it should take into account.

@emilymbender Talk about death panels. Not even a panel.

Unbelievable, except, all too believeable.

@emilymbender Small aside, but I unironically love the "my ability to even" turn of phrase. From the newsletter post, "paper-shaped objects" is a good one, too.

(I'm not qualified to say anything substantive about the post, but it was an interesting--if depressing--read. Thanks for sharing!)