This worst part is not the little mistakes, e.g. the wrong condition written on a form. Doctors make those mistakes too, and even though the LLM makes more of them, they're easily corrected.

The bigger issue is the focus problem. The LLM doesn't know what's important.

I'm currently seeing a doctor for a course of treatment. We check in monthly to see how the treatment is going, tweak my other meds, and talk about any issues that come up.

Because he is a normal human person, he starts every consult by asking me how I am and what's happening in my life. At our last consultation, I mentioned some life stress.

The LLM wrote in my consult notes that I was consulting him for the life stress. It doesn't even mention the condition I'm being treated for. Now that's in my notes, and when the doctor goes to review them (or if another clinic needs to look at them), they'll get completely the wrong idea about the consultation. The very serious medical condition is de-emphasised, and the inconsequential chatter is given pride of place.

I'm basically in the position of having to write my own consult notes for every consultation, and give a list of corrections to the doctor. It's another laborious task that makes medical care inaccessible.

E.g. "Researchers are using AI to find new drugs" means that they're probably running a program that's closer to a linear regression than a chatbot in its scale and architecture to extrapolate from previously characterized molecules to other areas of research

This does NOT mean that a chatbot has helped to cure cancer in any way

And anyone who uses this as a justification for hyperscale data centre investment, LLM adoption, or "AI" generally is being deceptive through equivocation on purpose