a beautiful bitchThis question is making me wonder how many people have already been straight up killed by chatgpt
scarlett
MH Thaung
Mister Softie 
@hannah it is depressing seeing it fill the gaps (massive) of our health system. Ugh
David Hopeward@hannah That is horrifying.
Caffeine Devil@hannah it did recently tell a recovering addict to have a little meth, as a treat, so,
Berkubernetus@hannah Of course, the counter question is: is it more people than have been killed by WebMD, Goop, et al?
@fuzzychef i see your point but neither of those will confidently tell you specifically the wrong number of mg to take
Jason Lefkowitz@hannah @fuzzychef WebMD, Goop et al also publish their misinformation on web pages, where everyone can see it. This at least creates the possibility that their misinformation can be spotted by third parties, and pressure applied to correct it.
Other than OpenAI, nobody sees what ChatGPT is whispering into the ears of vulnerable people
xenia :possum:@fuzzychef @hannah thing is, those sites just kinda vaguely say your symptoms could be X number of things without giving you any sort of concrete answers.
it runs you in circles that dont lead anywhere (good or bad) and ultimately are worthless without proper test to rule out stuff half of that stuff
it runs you in circles that dont lead anywhere (good or bad) and ultimately are worthless without proper test to rule out stuff half of that stuff
xXx_K1R4_xXx t3h FoXXXDerG oF d00m🏳️⚧️@hannah wheww man people's brains really are leaking out of their ears now huh
Hazel the Harridan@kirakira @hannah I think this is a symptom of a bigger problem: "... my doctor isn't responsive..." People are isolated. They don't know where to go for help, advice, and guidance. The entire human social fabric is falling apart and rather than go "wow, we should fix this!" the response is "fuck it, automate that shit."
0xC0DEC0DE07E9@hannah Sam Altman should be forced to use ChatGPT instead of going to any doctor. He keeps saying LLMs can replace doctors for “those who can’t afford it”
Princess Unikitty@hannah
Good data in, garbage out!
Good data in, garbage out!
BedastGPT@hannah I'm a mod in a large diabetes discord community. We have an info card literally telling people not to get medical advice from AI, and we especially discourage getting dosing advice from AI given insulin can be deadly when misused.
clewhttps://discord.com/invite/diabetes
It's an open and welcoming community, and is LGBTQ+ friendly.
Being very cautious -- is it okay to publish this link on the public website in question? I know here is kind of public, but Ask Metafilter gets indexed pretty heavily.
I'm not a Discord user and don't know if that's what "open" means.
Final stupidity, I went and posted it and then worried and had it taken down. But I can post it again! Or just put in a private message to the original poster.
@clew It's a completely public community. Feel free to post the link anywhere!
So long as the rules are followed (and they ARE enforced pretty aggressively), anyone is welcome with the understanding that this is a support community. We do gatekeep it to a degree, with a healthy chunk of the server hidden behind a manually assigned role.
garland@hannah I feel for this person. I was diagnosed with type 1 diabetes and started on long- and short- acting insulin in February. It is absolutely an overwhelming experience, and I can imagine how someone might resort to a LLM for help. But really this seems like more of an indictment of this person’s doctor/care team. You don’t just hand someone two types of insulin and a glucometer and shoo them out the door. #diabetes
splig
zompus
.vad//hakara🧭@hannah ChatGPT is our bloodletting bowl
Neville Park@hannah lmao at the comment that was "You need Baldur Bjarnason at least as much as you need insulin."
(cc @baldur)
allison@hannah it makes me so sad. lonely people desperate for help who are being failed by every single system
@aparrish it’s hard to think of a problem in america that widespread llm adoption doesnt make worse, but at least we were able to summarize some emails
Alexa Devreux-Swift@hannah yikes 😬
Jenn DolariAs someone who has been battling Type 2 Diabetes with two doctor, two endocrinologists, and has 24 hour monitoring of blood sugar...JESUS FREAKING CHRIST NO. NO NO NO NO. GO TO YOUR DOCTOR AND ASK FOR A CORRECTIONS SHEET. OH SWEET CRISPY WALNUTS DON'T USE CHATGPT FOR SOMETHING LIKE THIS.
aismallard@hannah @puppygirlhornypost2 There has already been half a dozen articles featuring people who killed themselves or caused serious harm to themselves as a result of going in really deep with these things.
It may be pointed out that some people are already liable to go down a rabbit hole and ChatGPT is just what they used, but we should also consider how these kinds of systems enable that, like with how they are natural at "yes, and"-ing.
This is also happening because of wider societal alienation and holes in support from lack of sufficient medical care and other kinds of expertise because of capitalism reasons. It's not great.
It may be pointed out that some people are already liable to go down a rabbit hole and ChatGPT is just what they used, but we should also consider how these kinds of systems enable that, like with how they are natural at "yes, and"-ing.
This is also happening because of wider societal alienation and holes in support from lack of sufficient medical care and other kinds of expertise because of capitalism reasons. It's not great.
paru Montgomery-Gator-svn@hannah a hacker saying "A computer cannot find out, therefore a computer must not fuck around" GPT cannot be held responsible, so you must not trust anything that it says because if it's wrong it's your fault for trusting it. Even if you give in to using it, you have to check everything it says against authoritative sources.
robcorneliusHe might as well have rolled some dice to decide what to do
Shijikori
R@hannah this is just the latest round of what happens when the medical system fails to meet people's actual needs (esp. when those needs are social/interpersonal and not merely related to access to specific molecules)
similar has happened before with women's issues, chronic illness, etc. getting consumed by quackery, MLMs, antivax, etc.
Bel Tamtu
blueberik
Andrey Rys 🇨🇿@hannah pinging @davidgerard this is what actually I was horrified with when my friend consulted chatgpt about his chronic pain and ended up being nearly suicidal
George Liquor, American@hannah jfc it doesn't get "confused"
Please please please stop treating it like a mind
Patch Arcana@hannah I'm rapidly approaching the point of "if you do stupid shit with a gun and die as a result, the gun didn't kill you" when it comes to LLM misuse.
To try and counter that impulse, I'm working on a handout/zine on why LLMs aren't "truth engines"
keirFox
groff@hannah Dangerous medical advice. Dangerous hiking advice.
There won't ever be a count of how many LLMs have endangered or killed.
https://m.ai6yr.org/@ai6yr/114716031622310118
There won't ever be a count of how many LLMs have endangered or killed.
https://m.ai6yr.org/@ai6yr/114716031622310118
AI6YR Ben (@ai6yr@m.ai6yr.org)
The Trek: They Trusted ChatGPT To Plan Their Hike — And Ended Up Calling for Rescue "Two hikers were rescued this spring from the fittingly named Unnecessary Mountain near Vancouver, Canada after using ChatGPT and Google Maps to plan their route and finding themselves trapped by unexpected snow partway up..." h/t @researchbuzz@researchbuzz.masto.host https://thetrek.co/they-trusted-chatgpt-to-plan-their-hike-and-ended-up-calling-for-rescue/ #hiking #SearchAndRescue #AI #LLMs #SAR
ferricoxide@hannah
Advice #1: get a new doctor.
Advice #1: get a new doctor.
Charlotte Walker
(❍ᴥ❍ʋ) ᚦᛟᚱᚾ (no kings please)@hannah people are getting psychosis from it for sure.
aproitzHoly Shit !
Don't ever trust #ChatGPT in suggesting the right #medication on #serious and possible #dangerous #medicine. Change the fucking #Doctor, if he is not #responsive and tell him this before. It is YOUR #Health an not the Doctor's.
Joshua Barretto@hannah Christ. It reminds me of this recent YouGov poll from the UK that showed that the general public has pretty much a perfectly inaccurate sense of what LLMs are and aren't actually good for. Pretty terrifying.
C++ Wage SlaveYeah, but a lot of British newspapers print horoscopes. As a nation, it seems we're easily led. 😟
Daniel BrotherstonTo be fair, the general public has a very very bad track record of who to trust on lots of these questions. People listen to celebrities, confident shysters, etc.
Dan Shuman@hannah Nightmare fuel.
crispycat 
chmod 777 Kayfox