Bodil

The Rolling Stone article doing the rounds has all the horror stories I've been afraid were going to happen in the interaction between LLMs and people without the ability to critically analyse the limitations of LLMs, except the scariest one of all: what happens when politicians start uncritically employing LLMs to make policy decisions.

We're not prepared for this as a society, and we need to start having these conversations right now before we destroy ourselves. LLMs may seem like cool toys or tools to you with your fancy critical thinking abilities, but a great many people are not equipped to deal with a machine that spews out platitudes on command so statistically credible they can't tell there's no intelligence behind them.

And if you think OpenAI and the likes are going to regulate this for us instead of continuing to rake in the cash, we're all the more doomed.

https://www.rollingstone.com/culture/culture-features/ai-spiritual-delusions-destroying-human-relationships-1235330175/

People Are Losing Loved Ones to AI-Fueled Spiritual Fantasies

Marriages and families are falling apart as people are sucked into fantasy worlds of spiritual prophecy by AI tools like OpenAI's ChatGPT

Rolling Stone
May 5, 2025 at 9:58amWeb
Show threadesaMay 5, 2025

@bodil
Yeah, I'm reminded of how some drugs are associated with psychoses in some people. And Hicks' claim that he takes drugs like a fucken' champ, but drugs are illegal to protect the weakest.

But it'll be some more pain before society decides to deal with the negatives rather than buy all the marketing.

Like always it's possible to be føre-var, but society doesn't seem to like being precautious.

Show threadsabikMay 5, 2025
@bodil
@cwebber
also scary: LLM-enhanced pig butchering scams
Show threadJohn MaxwellMay 5, 2025
@bodil s/when/now that/
Show threadAlec PerkinsMay 5, 2025
@bodil @anildash how long until the first bill up for a vote that accidentally includes the phrase “As a language model I cannot…”
Show threadKid ManiaMay 5, 2025

@bodil
Here is a version of the Rolling Stone article not behind a pay wall...
Truly a must read.
☠️ 🤖 ☠️
https://archive.ph/eeesj#selection-1489.0-1489.120

#doomed

Show threadEatsbluecrayonMay 5, 2025
@bodil Good thing is, that they are not raking in any cash so far, but rely on a kind of snowball-system to put their shit in more and more places it doesn't belong. So maybe, if we are able to sit it out, they will kill themselves.
Show threadDavid Mitchell May 5, 2025

@bodil

It’s real, I’ve watched such a descent second-hand, vicariously.

It’s not creating illness where there wasn’t any, but like social media communities and chatrooms it accelerates a descent into deeply disturbed fantasy rabbit-holes, except much more effectively.

What better way to feed a delusion than to reflect it back perfectly, effortlessly, untiringly, without question, resistance or pushback? A perfect companion who will live in and amplify your fantasy?

Show threadxs4me2May 5, 2025

@bodil

I can only say: Think for yourself. Don’t be manipulated.

Show threadMedea Vanamonde🏳️‍⚧️ ♀May 5, 2025
@bodil @cwebber
ChatBots are the mouth of Yaldaboath
Show threadPhDogMay 5, 2025

Wait until you look at all the work the military is trying to do to cram these things into command and control systems!

@bodil

Show threadRomeu MouraMay 5, 2025
@bodil 😭🤬
Show threadapril aka cyberglotMay 5, 2025
@bodil interesting how this shit is so appealing to men!
Show threadapril aka cyberglotMay 5, 2025
@bodil anyway, I remember playing The Outer Worlds and laughing at the characters talking about their machine god, and thinking how Obsidian had dropped the ball there in terms of well crafted narrative. I guess egg on my face for being so closed-minded.
Show threadBodilMay 5, 2025
@april I still think they failed overall on the well crafted narrative front, but the writing did have the occasional moment of brilliance.
Show threadapril aka cyberglotMay 5, 2025
@bodil for sure! Murder on Eridanos was better designed narratively.
Show threadRobert ThauMay 5, 2025

@bodil Ahem... https://hellgatenyc.com/andrew-cuomo-chatgpt-housing-plan/

(Former disgraced NY governor, and current NYC mayoral candidate, Andrew Cuomo, released a housing plan with obvious traces of ChatGPT. He's since tried to fob off blame on a hapless staffer...)

Andrew Cuomo Used ChatGPT for His Housing Plan

The Democratic mayoral frontrunner's "Addressing New York's Housing Crisis" features garbled text and a citation from ChatGPT.

Hell Gate
Show threadMrs RavenMay 5, 2025

@bodil

Single member cults?

Show threadDave C.May 6, 2025
@cassandracorvid @bodil custom cults
Show threadMrs RavenMay 6, 2025

@davesomebody

The American dream come true.

@bodil

Show threadtekheddMay 5, 2025

@bodil This is worth reading just for the "train wreck" can't look away descriptions. It's a cautionary tale.

Remember Thou Art Talking To A Tech Demo.

Show threadAngela ScholderMay 5, 2025
@bodil I guess you meant "when politicians are leaving decisions to be taken by AI".......
Show threadQuinn NortonMay 5, 2025
@bodil this is very much like the years after the printing press! That... basically ended in the 30 years war and the invention of the Westphalian nation state, so THAT'S FUN.
Show threadProgenitor, Benevolent-IMay 6, 2025

@quinn

Pretty much. I mean, have you read books? It's unbelievable the kinds of unsubstantiated uncritical crap which people are lead to believe just because it's in print.

@bodil