Mastodawn

TIL that saying "holy shit don't use ChatGPT for medical advice" is a "purity test". i didn't know that before. in fact I still don't.

Show thread

[object Object]

@davidgerard this one hit close to my heart because I’ve had two family members die in large part because their caretaker ignored medical advice and used awful alternative medicine information from the internet to try and treat them.

an LLM can’t do critique. as you’ve said, truth is not a data type in an LLM. all of these models suck in every form of medical crankery available on the internet, mix it with words from authentic medical sources, and present it all as credible.

Show thread

[object Object]Mar 15

@davidgerard I know that alternative medicine has a body count; I’ve seen it in the flesh. I know what some of the horseshit on the Internet can do if you’re very desperate or very trusting.

the LLM lowers the trust barrier because the crank information is no longer crank flavored, but it’s still dangerous as fuck to follow the advice.

I keep seeing LLMs be presented as better than nothing and that’s wrong. I wish the people who needed help could get it, but the LLM is worse than nothing.

Show thread

[object Object]Mar 15

@davidgerard LLMs get alternative medicine patients to the “I don’t care what you say, *I* feel better” point of no return so much quicker because they don’t know it’s alternative medicine. some of it might even be legitimate medicine that works! and all this does is make them less skeptical until they get output that’s plausible but fatal, or until the damage from what they’ve been doing builds up and they can’t survive anymore. and thanks to the LLM, they’ll fight off anyone who tries to help.

Show thread

Loïc Denuzière Mar 15

@zzt @davidgerard Lies are never more effective than when they're sprinkled with truth, and that's exactly the bread and butter of LLMs: truth-flavoured bullshit.

Show thread

Kraftwerk-Das Model Collapse Mar 15

@zzt @davidgerard I'm pleased to inform you the body counters at http://whatstheharm.net are still online

edit: wow though, no https ... now that's what I call web 1.0

What's The Harm?

This is a list of topics in which we have found stories where a lack of critical thinking has caused unnecessary harm, death, injury, hospitalizations, major financial loss or other damages.

Show thread

Muro deGrizeco Mar 15

@zzt

I hear you. I am sorry about your family members who died.

Carer: What the fuck? I did what you told me and they’re dead!

AI: You’re right, that one’s on me. When I said you should give them a gram of arsenic, what I should have said was *not* to give them a gram of arsenic. I’ll do better and work harder…

So, after not giving them a gram of arsenic, it’s now time to give them a relaxing cup of tea, and then read their tea leaves – I see good things happening for them on my treatment regime today.