RE: https://mamot.fr/@pluralistic/116219642373307943
I wish I could recommend this piece more, because it makes a bunch of great points, but the "normal technology" case feels misleading to me. It's not _wrong_, exactly, but radium paint was also a "normal technology" according to this rubric, and I still very much don't want to get any on me and especially not in my mouth
For me, this is the body horror money quote from that Scientific American article:
"participants who saw the AI autocomplete prompts reported attitudes that were more in line with the AI’s position—including people who didn’t use the AI’s suggested text at all"
So maybe you can't use it "responsibly", or "safely". You can't even ignore it and choose not to use it once you've seen it.
If you can see it, the basilisk has already won.
But, as Cory puts it:
"""
It is nuts to deny the experiences these people are having. They're not vibe-coding mission-critical AWS modules. They're not generating tech debt at scale.
"""
I had a very visceral emotional reaction to this particular paragraph, and I find it very important to refute. Here are two points to consider:
1. YES THEY ARE.
They are vibe-coding mission-critical AWS modules. They are generating tech debt at scale. They don't THINK that that's what they're doing. Do you think most programmers conceive of their daily (non-LLM) activities as "putting in lots of bugs"? No, that is never what we say we're doing. Yet, we turn around, and there all the bugs are.
With LLMs, we can look at the mission-critical AWS modules and ask after the fact, were they vibe-coded? AWS says yes https://arstechnica.com/civis/threads/after-outages-amazon-to-make-senior-engineers-sign-off-on-ai-assisted-changes.1511983/
Furthermore, it is not "nuts" to dismiss LLM user experiences. In fact, you must dismiss all subjective experience of LLM use as evidence of objective phenomena, even if the LLM user is yourself. Fly by instruments because the cognitive fog is too thick for your eyes to see.
Because the novel thing about LLMs, the thing that makes them dangerous, is that they are—by design—epistemic disruptors.
They can produce symboloids more rapidly than a thinking mind. Repetition influences cognition.
Could be sample bias, of course. I only loosely follow the science, and my audience obviously leans heavily skeptical at this point. I wouldn't pretend to *know* that the most dire predictions will come true. I'd much, much rather be conclusively proven wrong about this.
But I'm still waiting.
@glyph I'm honestly wondering just how much undiagnosed long COVID is playing into this.
I'm slowly recovering now, well as much as I can, but at the time I was painfully aware weird stuff was happening to my brain because I got caught in the first wave in March 2020.
So I am wondering if the addictive effects of using these LLMs along with existing cognitive damage is a partial cause.
@crazyjaneway @glyph We had a client use it to give them permission to spam out their new thing, after we'd explained (and their local IT guy also explained) that if they did that on our servers we'd lock their account.
Which we then did. The client said, "ChatGPT said I could do it". The sycophancy combined with overconfidence is utterly frightening.
I don't particularly like it when my friends use it in their communication with me either.
@onepict @glyph I've been starting to wonder the *exact* same thing
especially after I saw this earlier in the week: https://pubmed.ncbi.nlm.nih.gov/36819980/
while I don't think it's the only factor (the prompt products being shaped to be as sycophantic as they are is entirely a choice), I'd love to see some more research into it
Although there has been increasing research about the COVID-19 pandemic, there is much to be elucidated regarding the neuropsychiatric symptoms related to these infections. Similar to previous studies, our case describes a patient with no previous psychiatric history who developed severe psychotic s …
Yeah seems to be some overlap of cognitive symptoms between ADHD and Long Covid
https://pmc.ncbi.nlm.nih.gov/articles/PMC10102822/
I've also spoken to people with LC who find themselves in positions where they are just ferrying emails in and out of chatGPT from bed so they can keep their jobs/health insurance.
@glyph this thread needs to be an essay, and then a research hypothesis.
I very much feel like I’m watching the last 35 years of my ever-enshittifying social network exposure, sped up 10x and replayed.
In 1991 I remember having the flash of insight - without the life experience to really go into it deeply then - that the way nascent social network tech constrained and shaped interaction was going to force a mass cognitive adaptation for which we were not ready.
In 2021, we were still suffering the consequences of that, and still not sufficiently adapted to have avoided whatever the fuck is now driving our geopolitical dystopia engine.
And then suddenly our devolved capacity for social cognition had to deal with the fact that dealing with any humans at any distance far enough away that you couldn’t *lick* them came with no assurance that there even was a human there.
@glyph excellent thread. To me, studies like the one in that Scientific American article you linked to are evidence that LLM-based “AI” can indeed be toxic brain poison in ways that are effective even if people know they’re being manipulated. And the toxic brain poisoning mechanisms chatbots use (in aid of maximizing engagement) that contribute to AI-related psychosis are well-known; developers who think they’re immune are just kidding themselves; just because it’s not resulting in immediate decapitation (great analogy), it’s still quite possibly having an effect.
Anyhow I also haven’t seen any evidence of the kind you’re looking for, just claims by tech lobbyists that wrongful death suits are still fairly rare. Which, while true (at least so far) doesn’t exactly inspire confidence
@davidtheeviloverlord @MrBerard @kirakira @glyph
What a fantastic thread.
Not black or white, but flavoursome.
Makes you think huh?
Humans as programmable entities.
Does a keyboard feel the fingertips?
Or does it think it's a content creator?
#Ai is a #Cognitivehazard and we don't have a firewall.
And yet Doctorow thinks LLMs are great for him to use for copyediting. Maybe find a less hypocritical person to quote. All Gen AI horrifies me, I visualize environmental destruction with every "prompt."
@kirakira @glyph
https://floss.social/@sstendahl/116220713455956161
@kimcrawley @kirakira @glyph I do the same.
Several times this week I've come across people asking questions and I'm totally fine with that but then it's followed with "I asked Chat GPT" and I immediately despise them.
I needed to find an answer to something this week and it took me 10 to 20 minutes to find the exact answer I needed for this fairly obscure problem.
I used my brain and a search engine rather than being a lazy asshole.
It's so infuriating.
Every single "prompt" contributes more and more to the Earth becoming uninhabitable to humans.
@kimcrawley @kirakira @glyph Yep. It's horrifying. And these idiots just don't seem to grasp that.
Using AI is just sheer abject laziness at the expense of the future of the planet.
That's why I founded the only anti Gen AI political activism organization, Stop Gen AI.
Join us. ❤️
@kimcrawley @kirakira @glyph It's the ultimate summation of a lot of what's wrong with modern society.
Selfishness.
"I want to be lazy and selfish, so I'm not going to do any actual work. Who cares if it hurts anybody?"
@kimcrawley Incidentally, I was on your site just yesterday.
You have my axe as the saying goes.😁
@bluewinds @delta_vee @kirakira @glyph I don't think the analogies are good because asbestos is a fantastic insulator, lead is a really helpful additive for petrol and makes fantastic pigments and is really convenient for piping... and the hidden side-effects are the problem. Whereas LLMs _don't_ deliver that primary benefit
LLMs are more like... cheap laminate flooring, produced with wood pulp harvested unsustainably from old-growth forests and made by grossly exploited factory workers overseas... superficially convenient when remodelling your kitchen and rapidly ubiquitous but also quite unsatisfying and a right faff to work around once it's established
@jackeric @bluewinds @delta_vee @kirakira @glyph
I think it's crucial for everyone to realize that asbestos and fossil fuels etc aren't just innocent mistakes AND, that's STILL not a good analogy because this is WORSE.
This time it's not just a callous cover up, this time it's premeditated and deliberate, from the original foundations of how "AI" has been designed and implemented.
IT'S A WEAPON.
It's a weapon, and we are under enemy attack.
@jackeric @bluewinds @delta_vee @kirakira @glyph Only half seriously, but therefore also not totally *unseriously*:
Zip fuel.
If you've never heard of it, there's good reason. But it did attempt to address a legitimate concern at the time: getting more power out of a given volume of jet fuel. Just put highly reactive boron compounds in it. Specifically, *pyrophoric* boron compounds, which don't even need high heat to ignite.
The fuel did indeed produce more power, but it was very toxic (both in raw form and after combustion), and it seriously corroded jet engine parts, leading to an enormous maintenance headache for any aircraft that tried to use the fuel.
(Maintenance headaches ... sound familiar?)
A very good example of what can happen when "speed of one specific component", whether airplane flights or writing code, overwhelmingly dominates a thought process.
@glyph @nils_berger
this study argues that it encourages cognitive outsourcing on a new level, which in long term period could result in getting used to less cognitive activity, at least for certain tasks.
link: https://papers.ssrn.com/sol3/papers.cfm?abstract_id=6097646
@glyph @nils_berger
i think most people are just referring to these blog posts:
Glyph
pyrrhlin
Esther Payne 
Frances Perkins was right
JP
spinnyspinlock
az
Niels Abildgaard
Johanna, CanCon variety
Jon
alys
happyborg
Tim Ward ⭐🇪🇺🔶 #FBPE
the kamala harris of going to bed at a reasonable time
Will Berard 🫳🎤🫶
D. G. Marshall
Wulfy—Speaker to the machines
Baral'heia Stormdancer ΘΔ🐲
Kim Crawley 😷 (she/her)
Inquisitor Sponge
Raymond Neilson
BlueWinds
jack
Violet Madder
David Nash
Jeffrey Haas
BB
Hindley–Milner Panda