mikeceMar 19

A rogue AI led to a serious security incident at Meta

https://www.theverge.com/ai-artificial-intelligence/897528/meta-rogue-ai-agent-security-incident

A rogue AI led to a serious security incident at Meta

Last week, an AI agent similar to OpenClaw triggered a high-severity security incident at Meta by independently giving inaccurate technical advice on an employee forum.

The Verge
Show threadUhhrrrMar 19
The two errors, then, were that the LLM hallucinated something, and that a human trusted the LLM without reasoning about its answer. The fix for this common pattern is to reason about LLM outputs before making use of them.
Show threadsomewhereoutth
However - Automation bias is a common problem (predating AI), the 'human-in-the-loop' ends up implicitly trusting the automated system.
Mar 19 at 7:25pmWeb