jcoglanif I have to hear the term "prompt injection" one more time
you cannot have an "injection attack" in a system with no formal distinction between data and instructions. what you actually have is an "everything is instructions" model and a failure to isolate untrusted inputs from the elevated privilege of access to private information
this is not a novel or surprising means of attacking systems. of course, obviously, if you give a system with no formalisable behaviour, that may execute anything as an instruction by design, elevated privileges and untrusted input, this will happen
the category error here is deploying something with unformalisable behaviour with any privilege to do anything at all without the express confirmation of a trusted human operator
the way people are deploying LLMs is driving a freight train through the principle of least privilege and being surprised at the results
Glyph@jcoglan I think the original insight was good but the term has begun to chafe and your explanation is absolutely on point as to why. We need a punchier term that encapsulates this understanding but it’s hard to describe as an “exploit” or a “vulnerability class” when it’s just a total, comprehensive, catastrophic misunderstanding of the entire concept of security. With MCP shit rolling out at a breakneck pace we are truly entering the time of monsters