Tee Rex 🏳️‍🌈🦖2d ago

Something a bit worrying to note about using Ai in healthcare.

I’ve had two specialist appointments recently, both using ai to transcribe. Both sent report letters with inaccuracies about my diagnoses and past medical history. Even my GP was like, “huh, that directly contradicts what I put in the referrals.”

I have followed up both and requested amendments (which were done) but if I hadn’t, these inaccuracies could have significantly damaged ongoing care, further treatment or insurance claims.

Human error has always been a factor, but both doctors were clearly using the ai software and assuming what it spat out was correct. They made no other notes during the appointments to cross-reference and double check. This is how Very Bad Things can happen.

Show threadTee Rex 🏳️‍🌈🦖
Two friends have told me in the last week they had similar issues happen. One had an incorrect diagnosis listed before they had a procedure done. The other noted viral not bacterial infection (although they did at least get the medication they needed). I feel like I’m being a pain in the bum going over everything and requesting corrections, but I’m seeing so many mistakes, to the point where any human reading them would immediately say “that doesn’t even make sense”. I worry for those who don’t or aren’t capable of checking these things. Sure, using ai might save the docs 10 minutes per patient in the ER but is that really worth the risks?
Mar 27 at 1:01amWeb
Show threadStevez2d ago
@bloodflowersburning Where are doctors using ai like this? And how did you check the transcribes?🤔
Show threadTee Rex 🏳️‍🌈🦖2d ago
@theron29 Genuine question or scepticism? I’m in Aotearoa NZ. Doctors were from two different specialist medical departments. Both used ai software to record consultation and take notes. The report letters sent to my GP contained multiple discrepancies about conditions discussed, and referenced in GP referrals. If they had checked before sending they would have realised mistakes had been made. My GP questioned the content, which was how I became aware. I can provide several specific examples but would rather not on a public forum to a stranger. However, both letters were re-assessed and sent again with corrections on request. Hope this helps.
Show threadStevez2d ago

@bloodflowersburning Genuine question (from central EU). (AI Scepticism is expected to come a bit later on... 🙂 ).
Doctors are not using AI here, yet. I guess this tech had to be certified and tested before it was admitted into doctor's realm?

Although this does not seem to be the worst usecase scenario where&how to use AI, your detailed explanation gives doubts whether the tech is actually ready now for anything *this* important.... 😏

Show threadTee Rex 🏳️‍🌈🦖2d ago

@theron29 Agreed. Not the worst case scenario. For me personally it could have caused issues with further treatment, getting reimbursed by insurance, and caused confusion when needing ongoing care with other providers. So more an avoidable inconvenience and extra paperwork rather than a dangerous outcome in this example. I hope that’s the worst possibility across the board, and that people check their notes carefully to catch any inconsistencies.

Mistakes in medical notes have always happened, unfortunately it’s inevitable. Only time will tell if this becomes more of an issue if/when ai transcription is used in medical settings more frequently and if it generates a higher number of errors as opposed to human note taking. What I think is essential is that we still retain a human buffer to assess factual accuracy, rather than simply assuming (hoping?) the software can do it better.

For more info, the software Heidi AI Scribes has been endorsed for use within Health NZ. https://www.tewhatuora.govt.nz/health-services-and-programmes/digital-health/generative-ai-and-large-language-models#naiaeag-endorsed-tools

Generative AI and Large Language Models - Health New Zealand | Te Whatu Ora

Building the future of health

Show threadCass🇨🇦14h ago
@bloodflowersburning it's amazing NZ would authorise something worse than simple voice to text transcription for doctor notes. But I'm old school, I still do searches and visit sites like Cleveland for medical guidance.
Show threadTee Rex 🏳️‍🌈🦖4h ago
@Cass_m I suspect NZ has long been a bit of a Guinea pig for these things. Good sample size with a manageable (isolated) but diverse population and a legal framework that often doesn’t fully protect its citizens, makes the data acquisition quite appealing to certain groups.
Show threadCass🇨🇦3h ago
@bloodflowersburning I how not. That's would make it a hostile government.
Show threadTee Rex 🏳️‍🌈🦖3h ago
@Cass_m it’s a bit of a long-perpetuated idea (myth?) that tech companies often conduct A/B testing on new products here. I can’t speak to the accuracy of that belief, and I’m not convinced it would make economic sense anyway mostly because of the complexities of large-scale research sampling.
Show threadAethon13h ago

@bloodflowersburning

As an ex healthcare person based in the UK, it’s being actively pushed all the time and of course we always implicitly trust a machine so the letters and notes are rarely checked.

The most tech I would use was dictation software for my letters for speed and even then they needed reading after transcription as it didn’t recognise certain terms.

This stuff scares me as decisions are made between professionals based on the contents of communications and I have heard so many tales like yours that we simply don’t know where the errors are creeping in.

The only way is to keep your own notes and records but that can be difficult and exhausting.

Good luck.