mikeceMar 19

A rogue AI led to a serious security incident at Meta

https://www.theverge.com/ai-artificial-intelligence/897528/meta-rogue-ai-agent-security-incident

A rogue AI led to a serious security incident at Meta

Last week, an AI agent similar to OpenClaw triggered a high-severity security incident at Meta by independently giving inaccurate technical advice on an employee forum.

The Verge
Show threadUhhrrrMar 19
The two errors, then, were that the LLM hallucinated something, and that a human trusted the LLM without reasoning about its answer. The fix for this common pattern is to reason about LLM outputs before making use of them.
Show threadSlinkyOnStairs

> The fix for this common pattern is to reason about LLM outputs before making use of them.

That is politics. Not engineering.

Assigning a human to "check the output every time" and blaming them for the faults in the output is just assigning a scapegoat.

If you have to check the AI output every single time, the AI is pointless. You can just check immediately.

Mar 19 at 7:26pmWeb