New: Researchers have quantified how easy AI search is to manipulate. Just 13 words buried in a random Reddit comment can poison AI search results. They suggest this is not easy to stop: "The way you can attack these systems is so much dumber than you think it is"

https://www.404media.co/it-is-trivially-easy-to-use-reddit-to-manipulate-ai-search-research-suggests/

It Is Trivially Easy to Use Reddit to Manipulate AI Search, Research Suggests

"We show that a tiny snippet—just 13 words—of retrieved text on a UGC website like Reddit, Wikipedia, Quora, or Facebook can change AI agents to output spam / scam content pretty consistently."

404 Media

@jasonkoebler
"LLMs export their trust to external content moderation strategies that exist on sites like Wikipedia or Reddit or Quora or StackExchange. So these deep research systems are increasingly relying on the judgment and taste of subreddit moderators or Wikipedia editors, and at the same time those websites are increasingly under strain from people and companies trying to manipulate them.”

Killer quote there I think.

@krnlg @jasonkoebler

More ubiquitous are the countless people just wanting to poison the well for these LLMs. And I don't blame them.

When someone submits a resume, a thesis, a business plan, etc. that relied on the LLM doing all of the work, then I want it to be obvious to anyone with any brain cells.

If I see another resume from a 23 yr old that states they spent five years doing market research for Nestle or IBM on using Linux BTW....