LLM-generated EHR summaries can fail in the worst way: one confident hallucination that changes clinical meaning.
This guide shows claim-level evaluation, risk-weighted safety metrics, and production gates (generate → verify, conservative fallbacks, monitoring).
Read: https://codelabsacademy.com/en/blog/evaluating-llm-hallucinations-clinical-safety-ehr-summaries?source=mastodon
🩻 AI isn’t here to replace radiologists — it’s here to write their first draft.
🔗 Hybrid framework for automated generation of mammography radiology reports. Computational and Structural Biotechnology Journal, DOI: https://doi.org/10.1016/j.csbj.2025.07.018
📚 CSBJ: https://www.csbj.org/
#Radiology #Mammography #BreastCancerScreening #MedicalImaging #DigitalHealth #RadiologyAI #CancerDetection #SpanishAI #ClinicalNLP #HealthcareInnovation #GlobalHealth #HumanCenteredAI #AIinHealthcare #MedicalAI #HealthTech
🧠 Is AI ready to be your doctor’s second opinion — or is it still a black box?
🔗 From explainable to interpretable deep learning for natural language processing in healthcare: How far from reality?. Computational and Structural Biotechnology Journal, DOI: https://doi.org/10.1016/j.csbj.2024.05.004
📚 CSBJ Smart Hospital: https://www.csbj.org/smarthospital
#XIAI #ExplainableAI #InterpretableAI #HealthcareAI #NLPinHealthcare #Transformers #DeepLearning #ClinicalNLP #AIethics #MedicalAI #XAI #IAI
Starting today's #AMIA2024 sessions with the @NLP (#NaturalLanguageProcessing, #ClinicalNLP) Working Group Pre-Symposium Workshop. Felix Morales opened with a talk on "ML Pipeline Flags Instances of ARDS from EHR" (https://www.medrxiv.org/content/10.1101/2024.05.21.24307715v3).
I'll have a talk later in this session called "Approximate Randomization Technique
for Comparing Performance Between Demographic Subgroups" (Sample code: https://codeberg.org/HeiderLab/article-addenda)
Physicians could greatly benefit from automated diagnosis and prognosis tools to help address information overload and decision fatigue. Intensive care physicians stand to benefit greatly from such tools as they are at particularly high risk for those factors. Acute Respiratory Distress Syndrome (ARDS) is a life-threatening condition affecting >10% of critical care patients and has a mortality rate over 40%. However, recognition rates for ARDS have been shown to be low (30-70%) in clinical settings. In this work, we present a reproducible computational pipeline that automatically adjudicates ARDS on retrospective datasets of mechanically ventilated adult patients. This pipeline automates the steps outlined by the Berlin Definition through implementation of natural language processing tools and classification algorithms. First, we used labeled chest imaging reports from two different hospitals over three different time periods to train an XGBoost model to detect bilateral infiltrates, and a subset of attending physician notes from one hospital labeled for the most common ARDS risk factor (pneumonia) to train another XGBoost model to detect a pneumonia diagnosis. Both models achieve high performance when tested on out-of-bag samples—an area under the receiver operating characteristic curve (AUROC) of 0.88 for adjudicating chest imaging reports, and an AUROC of 0.86 for detecting pneumonia on attending physician notes. Next, we integrate the models and validate the entire pipeline on a fourth cohort from a third hospital (MIMIC-III) and find a sensitivity of 93.5% — an extraordinary improvement over the 22.6% ARDS recognition rate reported for these encounters — along with a false positive rate of 18.8%. We conclude that our reproducible, automated diagnostic pipeline exhibits promising ARDS retrospective adjudication performance, thus providing a valuable resource for physicians aiming to enhance ARDS diagnosis and treatment strategies. We surmise that real-time integration of the pipeline with EHR systems has the potential to aid clinical practice by facilitating the recognition of ARDS cases at scale. ### Competing Interest Statement The authors have declared no competing interest. ### Funding Statement Feihong Xu was supported in part by the National Institutes of Health Training Grant (T32GM008449) through Northwestern University's Biotechnology Training Program. Curtis H. Weiss was supported by the National Heart Lung and Blood Institute (R01HL140362 and K23HL118139). Luís A. Nunes Amaral was supported by the National Heart Lung and Blood Institute (R01HL140362). Luís A. Nunes Amaral and Feihong Xu are supported by the National Institute of Allergy and Infectious Diseases (U19AI135964). ### Author Declarations I confirm all relevant ethical guidelines have been followed, and any necessary IRB and/or ethics committee approvals have been obtained. Yes The details of the IRB/oversight body that provided approval or exemption for the research described are given below: Institutional Review Board of Northwestern University gave ethical approval for this work (STU00208049). Institutional Review Board of Endeavor Health gave ethical approval for this work (EH17-325). I confirm that all necessary patient/participant consent has been obtained and the appropriate institutional forms have been archived, and that any patient/participant/sample identifiers included were not known to anyone (e.g., hospital staff, patients or participants themselves) outside the research group so cannot be used to identify individuals. Yes I understand that all clinical trials and any other prospective interventional studies must be registered with an ICMJE-approved registry, such as ClinicalTrials.gov. I confirm that any such study reported in the manuscript has been registered and the trial registration ID is provided (note: if posting a prospective study registered retrospectively, please provide a statement in the trial ID field explaining why the study was not registered in advance). Yes I have followed all appropriate research reporting guidelines, such as any relevant EQUATOR Network research reporting checklist(s) and other pertinent material, if applicable. Yes The datasets analyzed in this study will be made available upon peer-reviewed journal publication at ARCH repository hosted by Northwestern University. The Python code to reproduce the reported results will be made available upon peer-reviewed journal publication at the Amaral Lab GitHub repository. <https://arch.library.northwestern.edu> <https://github.com/amarallab>
Piek Vossen will give a keynote presentation at the Dutch #ClinicalNLP workshop on 24 June
Keynote Speaker: Prof.Dr. Piek Th.J.M. Vossen Title: Developing Dutch Language Models and Conversational AI for monitoring functioning and wellbeing Abstract: In this talk I will describe how we developed a Dutch Large Language Model from millions of medical notes and fine-tuned the model to track the functional level of patients over time according to the WHO International classification of functioning (ICF). I will also present our recent work to build…
I'll share the top-ranking results of our team "Language and intelligence(LAILab) machine learning department at Moffitt Cancer center" on the chemotherapy timeline extraction sharedtask at #ClinicalNLP2024
Looking forward to participating in @naaclmeeting and particularly #ClinicalNLP workshop in Mexico City.
https://sites.google.com/view/chemotimelines2024/leader-board?authuser=0
Interested in #ClinicalNLP? Our colleague Tobias Mayer presented his research on Monday at #EACL2024 in 🇲🇹 !
Find out more about "Predicting Client Emotions and Therapist Interventions in Psychotherapy Dialogues" in this thread: https://sigmoid.social/@UKPLab/112115994096462953
Attached: 1 image Can NLP assist psychotherapy research? YES ✨ Our #EACL2024 paper on Clinical NLP will be presented today at 16:15 CET – read more about it in this 🧵(1/8) #Psychotherapy #ClinicalNLP #NLPforMentalHealth #NLProc
Consider to look up the authors Tobias Mayer, Neha Warikoo, Amir Eliassaf and Dana Atzil-Slonim (Psychotherapy Research Lab/Department of Psychology, Bar-Ilan University) and @Iryna Gurevych (UKP Lab/Hessian.ai), if you are interested in more information or an exchange of ideas. (8/8) See you in 🇲🇹!
#Psychotherapy #ClinicalNLP #NLPforMentalHealth #NLProc #EACL2024
We have encountered interesting challenges with regard to ambiguity in interpreting cultural slang in Hebrew. (7/🧵)
#Psychotherapy #ClinicalNLP #NLPforMentalHealth #NLProc #EACL2024
UKP Lab
Code Labs Academy
CSBJ
Paul M. Heider
Djoerd Hiemstra 🍉
Shohreh