Tee Rex 🏳️‍🌈🦖2d ago

Something a bit worrying to note about using Ai in healthcare.

I’ve had two specialist appointments recently, both using ai to transcribe. Both sent report letters with inaccuracies about my diagnoses and past medical history. Even my GP was like, “huh, that directly contradicts what I put in the referrals.”

I have followed up both and requested amendments (which were done) but if I hadn’t, these inaccuracies could have significantly damaged ongoing care, further treatment or insurance claims.

Human error has always been a factor, but both doctors were clearly using the ai software and assuming what it spat out was correct. They made no other notes during the appointments to cross-reference and double check. This is how Very Bad Things can happen.

Show threadjustin2d ago
@bloodflowersburning Ask GPT to write code for you. Same problem,. forgets or invents context, addresses a different problem, gets fixated on minutiae when the problem is structural. And if you even hint at what you think the Delphi 2.0
will fart out you get "CONGRATULATIONS YOUR A GENIUS" :/
Show threadTee Rex 🏳️‍🌈🦖2d ago
@nzJayZee sadly these are all phrases that I don’t have much frame of reference with, but I’m assuming the tl/dr is basically “ai hallucinates most of the content it spews out and generally makes things worse”?
Show threadjustin2d ago
@bloodflowersburning Yes. In my experience it does make things worse by inventing and obsessing over bad solutions (keeps returning to the same shitty solution to a programming problem) What keeps me up at night is algorithmic cruelty when it's used in stuff like job applications or negotiating social services., "Computer Says No"
Show threadTee Rex 🏳️‍🌈🦖2d ago

@nzJayZee quoted from this article on RNZ: “He said jobseekers were using AI to generate their applications, while employers were using AI to read them.”
The snake is eating its own tail.

https://www.rnz.co.nz/news/business/590746/jobseekers-and-advocates-disturbed-as-companies-screen-applications-with-ai

Jobseekers and advocates disturbed as companies screen applications with AI

Advocates say the use of AI to screen job applications is dehumanising and creates bias.

RNZ
Show threadjustin
@bloodflowersburning I like the idea of applicant pushback. for something like $40 NZD /mth you can have all the "job application agents" via Claude. Totally agree about the snake.eating its tail . We should be building community resilience instead of data centers IMO.
Mar 27 at 7:30pmWeb
Show threadTee Rex 🏳️‍🌈🦖1d ago
@nzJayZee careful now, “community” seems to be a dirty word in some circles. Don’t be that radical lefty reminding people to be kind and care for others. 😉
Show threadjustin1d ago
@bloodflowersburning I'd never! The market knows best.
Show threadjustin1d ago
@bloodflowersburning When you help someone with their groceries/stairs/anything or call an ambulance when someone's hurt the most important thing shouldn't be "How am I compensated". David Graeber called this (deliberately provocative) "baseline communism" it's why when 2 people working in a repair shop go "pass me the wrench," ..""ok" instead of entering into a wrench contract
Show threadjustin1d ago
@bloodflowersburning (I know that's not how NZ works, and I feel sad about it)