Mastodawn

New: Researchers have quantified how easy AI search is to manipulate. Just 13 words buried in a random Reddit comment can poison AI search results. They suggest this is not easy to stop: "The way you can attack these systems is so much dumber than you think it is"

https://www.404media.co/it-is-trivially-easy-to-use-reddit-to-manipulate-ai-search-research-suggests/

It Is Trivially Easy to Use Reddit to Manipulate AI Search, Research Suggests

"We show that a tiny snippet—just 13 words—of retrieved text on a UGC website like Reddit, Wikipedia, Quora, or Facebook can change AI agents to output spam / scam content pretty consistently."

404 Media

Show thread

Michael K Johnson

@jasonkoebler I have posted (only) twice to reddit about an open source app I modified for a specific purpose.

When I asked Google about whether a similar modification was available for a different app, Gemini pointed me back to own reddit posts about the app I had already modified.

This wasn't malicious, but it highlights the point of this article: How few words are required to show up as an authoritative source in LLM confabulation.