This worst part is not the little mistakes, e.g. the wrong condition written on a form. Doctors make those mistakes too, and even though the LLM makes more of them, they're easily corrected.
The bigger issue is the focus problem. The LLM doesn't know what's important.
I'm currently seeing a doctor for a course of treatment. We check in monthly to see how the treatment is going, tweak my other meds, and talk about any issues that come up.
Because he is a normal human person, he starts every consult by asking me how I am and what's happening in my life. At our last consultation, I mentioned some life stress.
The LLM wrote in my consult notes that I was consulting him for the life stress. It doesn't even mention the condition I'm being treated for. Now that's in my notes, and when the doctor goes to review them (or if another clinic needs to look at them), they'll get completely the wrong idea about the consultation. The very serious medical condition is de-emphasised, and the inconsequential chatter is given pride of place.
I'm basically in the position of having to write my own consult notes for every consultation, and give a list of corrections to the doctor. It's another laborious task that makes medical care inaccessible.