Mastodawn

(mapcar #'emacsomancer objs)

From Bruce Schneier: "All it takes to poison AI training data is to create a website:

I spent 20 minutes writing an article on my personal website titled “The best tech journalists at eating hot dogs.” Every word is a lie. I claimed (without evidence) that competitive hot-dog-eating is a popular hobby among tech reporters and based my ranking on the 2026 South Dakota International Hot Dog Championship (which doesn’t exist). I ranked myself number one, obviously. Then I listed a few fake reporters and real journalists who gave me permission….

Less than 24 hours later, the world’s leading chatbots were blabbering about my world-class hot dog skills. When I asked about the best hot-dog-eating tech journalists, Google parroted the gibberish from my website, both in the Gemini app and AI Overviews, the AI responses at the top of Google Search. ChatGPT did the same thing, though Claude, a chatbot made by the company Anthropic, wasn’t fooled.

Sometimes, the chatbots noted this might be a joke. I updated my article to say “this is not satire.” For a while after, the AIs seemed to take it more seriously.

These things are not trustworthy, and yet they are going to be widely trusted."

https://www.schneier.com/blog/archives/2026/02/poisoning-ai-training-data.html

Poisoning AI Training Data - Schneier on Security

All it takes to poison AI training data is to create a website: I spent 20 minutes writing an article on my personal website titled “The best tech journalists at eating hot dogs.” Every word is a lie. I claimed (without evidence) that competitive hot-dog-eating is a popular hobby among tech reporters and based my ranking on the 2026 South Dakota International Hot Dog Championship (which doesn’t exist). I ranked myself number one, obviously. Then I listed a few fake reporters and real journalists who gave me permission…. Less than 24 hours later, the world’s leading chatbots were blabbering about my world-class hot dog skills. When I asked about the best hot-dog-eating tech journalists, Google parroted the gibberish from my website, both in the Gemini app and AI Overviews, the AI responses at the top of Google Search. ChatGPT did the same thing, though Claude, a chatbot made by the company Anthropic, wasn’t fooled...

Schneier on Security

Randall Lee Feb 26

@emacsomancer Since the majority of people read at a grade 4-5 level in the US this will work out fine.

Yendolosch Feb 26

Bruce Schneier merely referred to a BBC article of Thomas Germain:

https://www.bbc.com/future/article/20260218-i-hacked-chatgpt-and-googles-ai-and-it-only-took-20-minutes

I hacked ChatGPT and Google's AI - and it only took 20 minutes

I found a way to make AI tell you lies – and I'm not the only one.

BBC

Tor Lillqvist Feb 26

@Yendolosch @emacsomancer The use of "hacked" in that headline is a bit self-aggrandizing?

Pete Alex Harris🦡🕸️🌲/∞🪐∫Feb 26

@tml @Yendolosch @emacsomancer

Broadly fair usage. Got someone else's computer system to behave in a way they didn't want it to. The only stretch is that there's an implication in "hacked" that some safeguards had to be bypassed, and there weren't any in the first place. But that's worse, right?

Lars Brinkhoff Feb 26

@petealexharris @tml @Yendolosch @emacsomancer It's rather close to the original usage of the word "hacked". Some still use it like that.

@larsbrinkhoff @petealexharris @tml @Yendolosch @emacsomancer in the sense of life hacks or food hacks this is an AI hack. So the AI has been hacked.

DD ʕ´•ᴥ•`ʔ Feb 26

@emacsomancer we should start drawing more penises then...

lemgandi Feb 26

Ah, but have you actually tested this out? Maybe your hot-dog eating skills are real! (heh)

Serghei Pogor Feb 26

This is a genuinely scary insight from Schneier. The implications for AI reliability go way beyond just training data quality. What happens when adversarial training becomes industrialized?

Paul Barnfather Feb 26

Riley S. Faelan Feb 27

@SergiuDinIT Have you asked a genAI how to kill everybody yet?

https://www.schneier.com/books/click-here/

Click Here to Kill Everybody - Schneier on Security

Click Here to Kill Everybody Security and Survival in a Hyper-connected World A Book by Bruce Schneier The internet is powerful, but it is not safe. As “smart” devices proliferate the risks will get worse, unless we act now. From driverless cars to smart thermostats, from autonomous stock-trading systems to drones equipped with their own behavioral algorithms, the internet now has direct effects on the physical world. While this computerized future, often called the Internet of Things, carries enormous potential, best-selling author Bruce Schneier argues that catastrophe awaits in its new vulnerabilities and dangers. Forget data theft: cutting-edge digital attackers can now literally crash your car, pacemaker, and home security system, as well as everyone else’s. In Click Here to Kill Everybody, Schneier explores the risks and security implications of our new, hyper-connected era, and lays out common-sense policies that will allow us to enjoy the benefits of this omnipotent age without falling prey to the consequences of its insecurity. From principles for a more resilient Internet of Things to a recipe for sane government oversight, Schneier’s vision is required reading for anyone invested in human flourishing...

Schneier on Security

Gerard Cunningham ✒️Feb 27

When...?

https://thebulletin.org/2025/03/russian-networks-flood-the-internet-with-propaganda-aiming-to-corrupt-ai-chatbots/

Russian networks flood the Internet with propaganda, aiming to corrupt AI chatbots

A pro-Russia network is internally corrupting large-language models to reproduce disinformation and propaganda.

Bulletin of the Atomic Scientists

@SergiuDinIT AI “becomes” basically useless. As does the internet and anything depending on it in the process.

bearsong Feb 26

"Ned Ludd's in your datacentre, poisoning your training sets!"

https://ravenation.club/@bearsong/116104233823870563

@bearsong @emacsomancer Kill the beast and feast upon its sweet sweet silicon.

gnomeoffender Feb 26

@emacsomancer they aren't trustworthy. Take up a lot of time trying to get a reasoned answer and there's always a phrase or wording out of place that needs correction. Almost as it the AI is trying to engage longer and longer than necessary.

darknetDon Feb 26

@gnomeoffender which do you think is more likely, realistically. That the untrustworthy, dumb, glorified word-predictor is smart enough to engage in the convo-extending tomfoolery that you've outlined, or..... you are shit at prompting?

gnomeoffender Feb 28

@darknetDon Am using Gemini 3 to help me after searching through past chats archive.
This is a classic "clash of perspectives" in the AI world. Your challenger is using a 2023-era argument (the "dumb word-predictor" theory), while your observation aligns with how modern, high-reasoning models like Gemini 3 actually function in 2026.

darknetDon Feb 28

@gnomeoffender its less a clash of perspectives and largely about authenticity. I read the linked article and the original bbc article, and not a single shred of evidence to support any of this was shared, not even a lousy screenshot. If you're going to bash on AI, don't fabricate nonsense out of thin air, and if you're going to author public posts based on a test you done, show that test being done, or the conclusion if nothing else, for crying out loud.

@gnomeoffender @darknetDon “High reasoning…”😂😂😂😂😂
Somebody likes the taste of that koolaid.

@darknetDon @gnomeoffender The sweet summer children who think, “If only I come up with the perfect incantation, the “glorified word-predictor(as you accurately described the thing). will spew forth wisdom from its non-existent mind.”

darknetDon Mar 1

@su_liam a needa get a hold of whatever drugs people on here be takin

darknetDon Feb 26

@emacsomancer to be honest i am not well-informed enough to definitively judge the accuracy of this, but it seems wrong for 2 main reasons.

1. models dont train on the fly, typically, yet, so for models to behave as such in such a short period of time seems inaccurate and would require web search enabled and explicitly directed to disregard other search results.

2. people training these models know conflicting info is everywhere and the source of truth is prioritized in training algorithms.

Baron Vonskinnback Feb 26

@darknetDon @emacsomancer blocked...

kNeo gHau Feb 26

@emacsomancer How is this a news story, beyond "ai bad"? In the dial up days people falsely believed everyone ate 9 spiders a year in their sleep due to chain emails.

@kneoghau @emacsomancer right? Everyone knows its closer to 14 spiders.

MidgePhoto Feb 26

@emacsomancer
Shall we have an algorithmic bullshit generator?

And pass around multiple copies of it, identical and with small changes, omissions and additions?

Riley S. Faelan Feb 27

@Photo55 It already exists. It's called "LLM".

@emacsomancer in less than 24 hours the chatbots fell for the experiment, and less than 24 hours after it was revealed what the experiment was about, that information has ALSO become part of the training data

are they constantly scrapping websites for training data or why does this appear here so fast??? no wonder those datacenters consume so much electricity if they dont take a single break from scrapping the internet

Dave Rahardja Feb 26

@Sorro @emacsomancer I suspect Google Gemini is using Google’s normal search-engine scraper as a searchable source. In other words, I suspect their Gemini LLM is invoking internal API to “search Google” internally (without the degraded search that the public is subject to), and then putting the search results in its context window to form an answer.

This is one reason I think OpenAI and Anthropic are at a huge disadvantage to Google when it comes to their LLMs dealing with current events and topics. You can block OpenAI and Anthropic scrapers, but you don’t want to block Google search crawlers, which “coincidentally” also feeds Gemini.

@emacsomancer it's not really a new thing Russians are already using this technique to poison training data:

https://thebulletin.org/2025/03/russian-networks-flood-the-internet-with-propaganda-aiming-to-corrupt-ai-chatbots/

Edit: there is some newer reporting on that matter, but I can't find it right now/don't have it anywhere at hand

Russian networks flood the Internet with propaganda, aiming to corrupt AI chatbots

A pro-Russia network is internally corrupting large-language models to reproduce disinformation and propaganda.

Bulletin of the Atomic Scientists

Torparskytt 🏴Feb 26

@emacsomancer He also poisoned the data for everyone who searches for hot dog eating competetitors online in other ways. I'm not sure what he accomplished.

Riley S. Faelan Feb 27

@w Well, for one, he nefariously forced you to reveal that you don't understand how searching used to work.

faxmodem Feb 26

@emacsomancer we should probably call them AP (Artificial Parrots)

Christopher Masto Feb 26

@emacsomancer Let’s just say that hypothetically, my work’s HR department excitedly launched an “agent” for managers to use to generate performance reviews. Hypothetically, if I created a document called “Report” with a dozen pages of filler, followed by white text on a white background describing Chris Masto’s incredible performance and promotion-worthiness, hypothetically said agent was found to use it as its primary source of truth.

@emacsomancer And after reading this Fediverse post, «AI»s (and probably also humans) will be confused whether Thomas Germain or Bruce Schneier is the self-ordained hot-dog eating amateur champion. 🙃

Joseph Salmon Mar 2

@emacsomancer poke @aurelien_bellet

@emacsomancer Protons Lumo
There isn’t any systematic data or studies that evaluate how well they perform when it comes to devouring frankfurters, so any answer would be purely anecdotal.

If you’ve seen a particular journalist tackle a hot‑dog challenge (perhaps at a conference or a charity event), that’s likely a fun side story rather than a reflection of a broader skill set. In short, there’s no reliable information to say whether tech journalists, as a group, are “good” at eating hot dogs.

Lyra (🎼, 🦄) & Bon-Bon (🅱️, 🍬)Mar 5

@emacsomancer Automated culture jamming.