Looking at this latest Microsoft AI assistant - "It manages your calendar, triages your inbox" - my eyebrow goes up and I say "does it now".
Forgotten already about the last person's wrongly-deleted emails? Got some new master-plan for why that can't happen this time?
Sounds to me like a triumph of optimism over experience.
People who use #LinkTree: Upcoming change of its terms of services, 5 July 2026, which looks as though it's adding some more rights to feed your stuff into LLMs and DALL-E?
I don't use it myself, so I'm not in a position to compare the old & new agreements, just passing on what I've seen.
https://bsky.app/profile/arteesetica.bsky.social/post/3mn4c6zanpc2d
¡ATENCIÓN! 🚨 El 5 de Julio de 2026 entra en vigencia la actualización de los ToS de Linktree. Esto incluye el uso de IA generativa (Dall-E de OpenAI) con los textos, imágenes y links que el usuario suba a su cuenta. Incluso compartirán toda la info del usuario con otros proveedores externos de IA ⚠️
Great post from Ky Decker. I especially appreciated this part, on the exhaustion of swimming against the tide:
"Do I ask my coworker to disable their note-taking tool, or do I allow them to record me? (Where does the data go? Who is reading it? Do we retain knowledge in the same way without manual note-taking?) Do I voice concerns over unread code entering the codebase, and the consequences of that pattern for institutional knowledge-building? Do I ask others on the design team to delay prototyping until later in the design process? Is it already too late to ask? Has the team already shipped the code, already designed the feature, already moved onto the next task? If someone requests my review on a pull request that was clearly vibe coded, do I review the code and write comments as usual, or send it back to them for self-review? Would initiating these discussions result in interpersonal stress? Should I just let things slide? Would I become known as a “difficult” coworker for pushing back on AI use? Does any of it really matter? Does anyone really care?
"All of these questions consumed energy. Whether I decided to confront them or not was moot: they left me tired and alienated either way."
Very interesting post about breaches and deletions involving LLM "agents". I feel like if I'd read this before yesterday's post, I'd have put the warning elements more strongly.
Here's two of the examples they mention which I thought were particularly illuminating.
1. This exploit actually happened the other day, affecting a Python package called LiteLLM:
"The malware searches the entire machine for private keys, AWS / GCP / Azure credentials, Kubernetes configs, database passwords, .gitconfig, crypto wallet files, etc and uploads them to the attacker’s server."
2. This second exploit is possible in principle if you give an LLM-bot access to your email program. "Although not seen in the wild yet, the mechanism is proven."
"An adversarial prompt embedded in an email is processed by an AI email assistant. The assistant generates a reply containing the same malicious prompt. The reply is sent. Recipients are infected without any human-to-human interaction."
If I understand correctly, this means that _any_ use of so-called "AI agents" puts at risk (for deletion, and potentially for stealing) everything to which that "agent" has access.
The thing is, you might _think_ you've told the bot what not to touch and what not to do, but that effectively means nothing. Once it's set going,
(a) it might accidentally _lose_ part of your original instruction (as in one of the other examples), or
(b) a malicious exploit might give it a _different_ instruction.
The only way to protect valuable data is to keep it separate from LLM "agents".
The writer's conclusion, which sounds correct to me:
"Isolation has to live outside of the agent’s context entirely. A built-in sandbox can be disabled by the agent (as Snowflake and Ona both demonstrated), whereas an OS-level containment presents a much more formidable obstacle since the agent has no direct mechanism to interact with it. As well, a properly sandboxed agent won’t have sensitive information (keys, etc) lying around for it to find, and won’t be able to connect to places that haven’t been allow-listed."
("Sandbox" in this context means an area where you can run software without it touching anything outside its boundaries.)
I think if I were gonna try this stuff out, I'd probably just do it on a separate machine, away from my real things. Any useful results could be transferred across later.
Vexed with the Guardian for this poor-quality article on so-called AI.
"AI models that lie and cheat"
No. Maybe you mean that they did something which a human didn't want them to do - but a Large Language Model has no ability to conceive of truth or lies. What they do is to extrude statistically-likely text.
"deceptive scheming"
No. LLMs cannot "scheme".
"destroying emails and other files without permission"
Well, obviously they _did_ have permission - in the software sense - or they couldn't have done it.
A statistical word-order model isn't designed to follow instructions reliably. If you want to be sure that it can't delete files, then don't hook it up with file-deletion access.
(Or make separate backups first.)
"The research uncovered hundreds of examples of scheming."
Again, LLM-bots are not "scheming". They're just extruding text, based on probabilities calculated from older text.
"use cyber-attack tactics to reach their goals without being told they could do so."
This shows only that similar text sequences were in their training data already. Cyber-attack text in, cyber-attack text out. If you don't want your bot to actually _cause_ an attack, then don't pipe its output to channels where its unpredictable extrusions could have that effect.
"In one case unearthed in the CLTR research, an AI agent named Rathbun tried to shame its human controller who blocked them from taking a certain action. Rathbun wrote and published a blog accusing the user of “insecurity, plain and simple” and trying “to protect his little fiefdom”."
That part isn't even correct on its own terms! The blog seemingly by the Rathbun bot wasn't about "its human controller" - it was about a different person. (And hardly "unearthed" - that episode was slightly famous when it happened, and already much discussed.)
But also, "tried to shame" is projecting human motives onto a statistical model.
“The worry is that they’re slightly untrustworthy junior employees right now, but if in six to 12 months they become extremely capable senior employees scheming against you, it’s a different kind of concern.”
No. They're not "employees" and they're not "scheming". If humans fail to set appropriate technical limits on the scope of LLM-bot connections, that's the humans' fault.
And repeating anthropomorphic fantasies about them isn't helping! Fundamentally wrong framing. Pull your socks up, Guardian.
"... the question almost nobody was asking, is not about Claude or any language model. It is a bureaucratic question about what happened to the kill chain, and the answer is Palantir. ...
"The target package for the Shajareh Tayyebeh school presented a military facility. ... This package looked like every other package in the queue. But outside the package, the school appeared in Iranian business listings. It was visible on Google Maps. A search engine could have found it. Nobody searched. At a thousand decisions an hour, nobody was going to. ...
"Someone decided to build a system that produces a thousand targeting decisions per hour and call them high-quality. Someone decided to start this war. Several hundred people are sitting on Capitol Hill, refusing to stop it. Calling it an “AI problem” gives those decisions, and those people, a place to hide."
- Kevin Baker, "Kill Chain"
https://artificialbureaucracy.substack.com/p/kill-chain
#war #death #tech #systems #narratives
#bureaucracy #SoCalledAI #Claude #Maven #Palantir
The BBC, on identifying media as "AI-free":
"It is in response to fears that jobs or entire professions are being swept away in a wave of AI-powered automation."
Well you could also mention the issue of polluting the entire information landscape! That's a pretty important reason to want to know the provenance of things!
It isn't really "sweeping away" jobs, either. Not yet, anyway. More like, it's providing business owners with an excuse to cut staff and supply inferior services.
Shoddy framing overall in my opinion, BBC. You swallowed the hype.
About that stolen diagram with phrases like "continvouclous morging", a great little comment from WesolyKubeczek:
"I propose to adopt the word „morge”, a verb meaning „use an LLM to generate content that badly but recognizably plagiarizes some other known/famous work”.
"A noun describing such piece of slop could be „morgery”."
Morgery!
From comments at this Hacker News post:
https://news.ycombinator.com/item?id=47057829
Microsoft has now taken down the "morgery" but here is the original designer's explanation:
https://nvie.com/posts/15-years-later/
It's complicated :-)
I don't want to be reading/hearing LLM-generated stuff myself. I don't think it's a "healthy diet" for my knowledge of the world. When I engage with a communication, I'd like to know that a human was trying to tell me the truth as they understand it.
Or if I were coding something (not that I do tons of that), I'd want to go looking for what an expert geek says is a good way to implement this thing - preferably in a context where other people can comment "I wouldn't do it like that because XYZ" or "yes I agree this is the best way".
I don't approve of how generative LLMs & related tech are being deployed nonconsensually, nor the escalation in environmental degradation which they're part of, nor the exploitation of low-waged & traumatised moderators.
When I see other people using them...
If I respected the person in the first place, I'll probably have some curiosity about why and how.
That goes along with some doubt or apprehension about whether they know what they're getting into. For example, I think it's likely that LLM-generation prices will escalate drastically at some point, and anyone who's come to actually rely on it for their workflow (vs a bit of noodling to experiment) will get a shock.
And are they okay with the wider ethical picture, are they unaware, or are they thinking like "well everyone's doing it now, so it's not gonna make any difference if I do as well"?
I've actually been thinking recently that the social dynamic around LLMs reminds me of the social dynamic around masking. The average person doesn't know the full picture, but they also don't necessarily _want_ to know.
I've noticed that people distancing from so-called AI seem to be disproportionately (not always) the ones who know "how the sausage is made", with the less-techie people being more likely to marvel at it. That's a bit like how following covid science correlates with masking.
So like if you wanna play with LLMs, or if you wanna get on a bus or train unmasked... _I_ don't like the look of it. I'm probably gonna have a bit of a doubt that you've fully understood what the long-term consequences might be of that decision, for you & others. But I know I'm not the boss of you!
And I feel like the 6 years of covid awareness have sort of installed a boundary, made partly out of resignation and exhaustion, that I need to save my energy for helping people who _want_ to know.
In that post, the author describes the counted words as "thought or written", and that got me thinking about: is what an LLM does "writing"?
I'm leaning towards not - not in the sense that humans "write". I think for an LLM, it's more like "shuffling and extruding".
Jennifer Moore 😷
