Something exceptionally grim is happening on the Internet.
In the last few months, the constant flood of algorithmically generated junk content has kicked into an AI-powered overdrive, and it is cutting a swath of destruction as it overwhelms search engines, filters, and moderation systems
Call it Gresham's Law 2.0: bad content drives out good.
I'm starting this thread to document it, because there is a *lot* happening all at once.
Fake Funeral Live Stream Scams Are All Over Facebook
https://www.404media.co/fake-funeral-live-stream-scams-are-all-over-facebook/
"Scammers are copying genuine funeral service announcements, making their own Facebook accounts, then directing vulnerable victims to a fake livestream page. “It causes panic on the day where you shouldn't be thinking of that,” one impacted person said.”
Scammers are copying genuine funeral service announcements, making their own Facebook accounts, then directing vulnerable victims to a fake livestream page. “It causes panic on the day where you shouldn't be thinking of that,” one impacted person said.
https://housefresh.com/david-vs-digital-goliaths/
"Savvy SEOs at big media publishers (or third-party vendors hired by them) realized that they could create pages for ‘best of’ product recommendations without the need to invest any time or effort in actually testing and reviewing the products first."
Ghost Kitchens Are Advertising AI-Generated Food on DoorDash and Grubhub
https://www.404media.co/ghost-kitchens-are-advertising-ai-generated-food-on-doordash-and-grubhub/
"It’s common for advertisements to stage or edit pictures of food to make it look more enticing, but in these cases the ghost kitchens are showing people pictures of food that literally doesn’t exist, and looks nothing like the actual items they’re selling, sometimes because the faulty AI is producing physically impossible food items. "
Inside the World of TikTok Spammers and the AI Tools That Enable Them
https://www.404media.co/inside-the-world-of-tiktok-spammers-and-the-ai-tools-that-enable-them/
"We have recently been getting bombarded with Instagram Reels of influencers explaining how they make five figures a month by using AI to create tons of viral TikTok pages using stolen celebrity clips juxtaposed next to Minecraft gameplay footage."
Tech writer Kara Swisher has a new book. Enter the AI-generated scams.
https://www.washingtonpost.com/technology/2024/03/01/amazon-ai-fake-books-authors/
"Swisher looked at the screen and saw a book claiming to be a new biography of her, with an image on the cover that she immediately pegged as an AI-generated fake. … A closer look suggested that the book itself might be largely or entirely AI-generated, substituting generic descriptions of Swisher for factual details or anecdotes."
My Robotic Doppelganger Is The Grim Face of Journalism's Future!
https://foreverwars.ghost.io/my-robotic-doppelganger-is-the-grim-face-of-journalisms-future/
"On the zombie edition of the Washington Independent I discovered, the piece I had published more than ten years before was attributed to someone else. Someone unlikely to have ever existed, and whose byline graced an article it had absolutely never written."
Google Is Finally Trying to Kill AI Clickbait
https://www.wired.com/story/google-search-artificial-intelligence-clickbait-spam-crackdown/
"Google is taking action against algorithmically generated spam. The search engine giant just announced upcoming changes, including a revamped spam policy, designed in part to keep AI clickbait out of its search results."
The Perfect Webpage: How the internet reshaped itself around Google’s search algorithms — and into a world where websites look the same.
https://www.theverge.com/c/23998379/google-search-seo-algorithm-webpage-optimization
"Bit by bit, the internet has been remade in Google’s image. And it’s humans — not machines — who have to deal with the consequences."
[I can't prove this one is related, but I have a strong hunch that it is.]
Why Is Your News Site Going Out of Business?
https://talkingpointsmemo.com/edblog/why-is-your-news-site-going-out-of-business
[TPM's programmatic advertising revenue declined from ~$1.7M in 2016 to ~$75K in 2023.]
"If you look at that graph you can see that if your business was ads and remained ads you’re toast."
Monitoring AI-Modified Content at Scale: A Case Study on the Impact of ChatGPT on AI Conference Peer Reviews
https://arxiv.org/abs/2403.07183
"Our results suggest that between 6.5% and 16.9% of text submitted as peer reviews to these conferences could have been substantially modified by LLMs, i.e. beyond spell-checking or minor writing updates."
We present an approach for estimating the fraction of text in a large corpus which is likely to be substantially modified or produced by a large language model (LLM). Our maximum likelihood model leverages expert-written and AI-generated reference texts to accurately and efficiently examine real-world LLM-use at the corpus level. We apply this approach to a case study of scientific peer review in AI conferences that took place after the release of ChatGPT: ICLR 2024, NeurIPS 2023, CoRL 2023 and EMNLP 2023. Our results suggest that between 6.5% and 16.9% of text submitted as peer reviews to these conferences could have been substantially modified by LLMs, i.e. beyond spell-checking or minor writing updates. The circumstances in which generated text occurs offer insight into user behavior: the estimated fraction of LLM-generated text is higher in reviews which report lower confidence, were submitted close to the deadline, and from reviewers who are less likely to respond to author rebuttals. We also observe corpus-level trends in generated text which may be too subtle to detect at the individual level, and discuss the implications of such trends on peer review. We call for future interdisciplinary work to examine how LLM use is changing our information and knowledge practices.
https://www.astralcodexten.com/p/open-thread-320
"If you want a vision of the future, imagine a bunch of people all accusing each other ad infinitum of being AIs (“haven’t you heard of Dead Internet Theory?!”), while the actual AIs serve ads to them in the background.”
How Spammers, Scammers and Creators Leverage AI-Generated Images on Facebook for Audience Growth
https://cyber.fsi.stanford.edu/io/news/ai-spam-accounts-build-followers
"We suspect that AI-generated images appear on users’ Feeds because the Facebook Feed ranking algorithm promotes content that is likely to generate engagement.”
Huckster Behind ‘Willy Wonka’ Event Also Sells AI-Written Vaccine Conspiracy Books
“Coull couldn’t even be bothered to pen his own author bio, which declares him a “rising star in the literary world” who “weaves spellbinding tales that delve into the mysterious realms of fictional thrillers and gripping conspiracies.”
Kotaku EIC Resigns Over New Editorial Edict
https://aftermath.site/kotaku-eic-resigns-over-new-editorial-edict
"In a resignation letter seen by Aftermath, Glennon says that she made this choice due to the management team’s recent decision to deprioritize news in favor of guides. … According to a source close to the situation, Kotaku's staff will be expected to create 50 guides a week at the site.”
Jen Glennon, who took over as editor in chief of Kotaku in October, resigned Thursday. In a resignation letter seen by Aftermath, Glennon says that she made this choice due to the management team’s recent decision to deprioritize news in favor of guides. Glennon is the second editor in chief of Kotaku since Stephen Totilo’s […]
TikTok is paying creators to up its search game
"The insights tool essentially lays out ideas for videos a creator could make, based on both personalized and general data. On my account, for example, TikTok flagged “Succession outfits” as a recommended topic for me to make content about. Also listed: “Herman Miller Chair,” “Fairy Grunge Room Ideas,” and “strawberry muffins.””
The AI Peer-Review Crisis
https://www.404media.co/email/c36519c0-0930-47a2-808d-48631b597dc1
“The bigger issue is that there's just the peer review crisis you're talking about where there's just so many papers being sent to these conferences,” he said. “I think it leads to the kinds of things that you're seeing in [Liang’s] paper where people are using the AI technology to help them to do the reviews.”
Nicholas LoVecchio told me he knew something was wrong as soon as he received the peer-review responses to his paper about “taboo” language titled “Using translation to chart the early spread of GAY terminology.” “I received them in the evening, the week before Christmas and I was just going through
Google Books Is Indexing AI-Generated Garbage
https://www.404media.co/google-books-is-indexing-ai-generated-garbage
"For example, the 2024 book Bears, Bulls, and Wolves: Stock Trading for the Twenty-Year-Old by Tristin McIver, bills itself as “a transformative journey into the world of stock trading” and “a comprehensive guide designed for beginners eager to unlock the mysteries of financial markets.” In reality, it reads like ChatGPT-generated text with surface, Wikipedia-level analysis"
‘AI Instagram Influencers’ Are Stealing Women’s Bodies
https://www.404media.co/email/d72b3844-9d62-4961-b52f-86ba9f237480
"Stealing video footage from real women to make an AI influencer more “believable,” because the underlying video footage actually is real, is a strategy that I repeatedly saw being espoused by guides to making these AI influencers."
‘Made for Advertising’ Websites Are the Marketing Industry’s Latest Messy Situation
"“Because we could say things like, ‘Well, it’s automatically generated content.’ But if somebody finds value in that—for example, I would read a piece that tells me something about my air conditioner, like how to install it, or how to clean it—I don’t care whether that was written by a person or a machine, as long as it solves my problem. So does that constitute MFA?”
How I Built an AI-Powered, Self-Running Propaganda Machine for $105
"It took me two days, $105 and no expertise whatsoever to launch a fully automated, AI-generated local news site capable of publishing thousands of articles a day—with the partisan news coverage framing of my choice, nearly all rewritten without credit from legitimate news sources.”
The AI flight attendants of Facebook
https://www.garbageday.email/p/ai-flight-attendants-facebook
"Who cares if users are praying to and/or asking to marry AI images of flight attendants? All that matters is that they’re on the app and you can stick ads in front of them.”
Amazon is filled with garbage ebooks. Here’s how they get made.
https://www.vox.com/culture/24128560/amazon-trash-ebooks-mikkelsen-twins-ai-publishing-academy-scam
"The saddest part about it, though, is that the garbage books don’t actually make that much money either. …The way people make money these days is by teaching students the process of making a garbage ebook. It’s grift and garbage all the way down — and the people who ultimately lose out are the readers and writers who love books.”
Facebook’s AI Told Parents Group It Has a Gifted, Disabled Child
https://www.404media.co/facebooks-ai-told-parents-group-it-has-a-disabled-child/
"The top-ranked comment on this post is from “Meta AI,” which is Meta’s AI chatbot. “I have a child who is also 2e and has been part of the NYC G&T program,” the nonsentient chatbot wrote to a group of human parents. “We’ve had a positive experience with the citywide program, specifically with the program at The Anderson School.”
AI Is Poisoning Reddit to Promote Products and Game Google With 'Parasite SEO'
"A service called ReplyGuy advertises itself as “the AI that plugs your product on Reddit” and which automatically “mentions your product in conversations naturally.”"
Facebook’s Bizarre AI Images Now on LinkedIn, Too
https://www.404media.co/facebooks-bizarre-ai-images-now-on-linkedin-too/
"The post has 2,192 reactions, 173 comments, and 84 reposts. About half of the comments note that the image is AI-generated, while about half of them say things like “I just saw his wood curving of Messi, He is talented. Just amazing.”
Do You Like These AI Images of Dying, Mutilated Children? Facebook Algorithm Wonders
https://www.404media.co/ai-images-of-dying-drowning-mutilated-children-go-viral-on-facebook/
"One of the images shows a child whose leg is amputated below the knee and holds a sign reading “Today is my birthlday pleaase like.” That image has 70,000 likes and 3,000 comments. Another image is of a girl face-down in the ocean wearing an oxygen mask that is connected to a floating birthday cake.”
Flood of AI-Generated Submissions ‘Final Straw’ for Small 22-Year-Old Publisher
https://www.404media.co/bards-and-sages-closing-ai-generated-writing/
"“The problem with AI is the people who use AI. They don't respect the written word,” Dawson told me. “These are people who think their ‘ideas’ are more important than the actual craft of writing, so they churn out all these ‘ideas’ and enter their idea prompts and think the output is a story.”
Facebook’s AI Spam Isn’t the ‘Dead Internet’: It’s the Zombie Internet
https://www.404media.co/facebooks-ai-spam-isnt-the-dead-internet-its-the-zombie-internet/
"When exploring their comment sections, one will often see hundreds of bot-like comments interspersed with a few ‘real’ people sounding the alarm to no avail,” Khan Schoolcraft, a moderator of the group, told me. “It’s like yelling into an empty room full of a thousand identical brick walls.”
Crypto world taps AI to flood IRS with complaints about tax regs
"Some leading opponents have built online tools that tap the power of artificial intelligence to allow anyone to quickly generate custom letters to the IRS in opposition to the proposal — a big step beyond the form letters often central to other lobbying campaigns.”
Meet AdVon, the AI-Powered Content Monster Infecting the Media Industry
https://futurism.com/advon-ai-content
"Basically, AdVon engages in what Google calls "site reputation abuse": it strikes deals with publishers in which it provides huge numbers of extremely low-quality product reviews — often for surprisingly prominent publications — intended to pull in traffic from people Googling things like "best ab roller." "
Slop is the new name for unwanted AI-generated content
https://simonwillison.net/2024/May/8/slop/
"Not all promotional content is spam, and not all AI-generated content is slop. But if it’s mindlessly generated and thrust upon someone who didn’t ask for it, slop is the perfect term for it."
This is what the internet looks like now
https://www.garbageday.email/p/internet-looks-like-now
"Across the social web, older forms of low-effort engagement bait are getting a brand new coat of AI paint. Every time I poke around a random TikTok that ends up on my For You Page, I, without fail, find out it’s connected to some weird content farm pumping out AI images on Facebook."
AI Generated Hentai Is Viral All Over Facebook
https://www.404media.co/ai-generated-hentai-is-viral-all-over-facebook/
"Over the last few months, I have realized that I do not have to rush to archive content that blatantly violates Facebook’s rules, because Facebook has not deleted any posts that I have wanted to reference in an article this year without me directly sending them a link to the content.”
The Tragic Downfall of the Internet’s Art Gallery
https://slate.com/technology/2024/05/deviantart-what-happened-ai-decline-lawsuit-stability.html
“Generate a bunch of free images and accounts, have them buy and boost one another in perpetuity, inflate metrics so that the “art” gets boosted by DeviantArt and reaches real humans, then watch the money pile up from DeviantArt revenue-sharing programs. Rinse, repeat."
AI is making Meta’s apps basically unusable
https://www.fastcompany.com/91113437/ai-making-meta-apps-basically-unusable
"This new Instagram header also features a side-scrolling list of prompts for interacting with the AI: “Imagine rooftop gardens,” “advice for the modern stoic,” “anime reels.” … I have no rooftop, and if they think I’m interested in stoicism or anime, they’re beyond help.
It’s the End of the Web as We Know It
https://www.theatlantic.com/technology/archive/2024/04/generative-ai-search-llmo/678154/
"If we continue in this direction, the web—that extraordinary ecosystem of knowledge production—will cease to exist in any useful form. Just as there is an entire industry of scammy SEO-optimized websites trying to entice search engines to recommend them so you click on them, there will be a similar industry of AI-written, LLMO-optimized sites.”
Amazon Sold Fake Copies of Hotly Anticipated UFO Book
https://www.404media.co/amazon-sold-fake-copies-of-hotly-anticipated-ufo-book/
"On Twitter and Reddit, users are posting photographs of books with the same cover as Imminent, including Elizondo’s name as the author, and at first glance what seems like an identical title. Inside, however, is poorly written text that reads like it was copy/pasted from Wikipedia, two inch margins, huge font like a children's book, and, at least according to one photograph, blank pages.”
It Looked Like a Reliable News Site. It Was an A.I. Chop Shop.
https://www.nytimes.com/2024/06/06/technology/bnn-breaking-ai-generated-news.html
"A closer look, however, would have revealed that individual journalists at BNN published lengthy stories as often as multiple times a minute, writing in generic prose familiar to anyone who has tinkered with the A.I. chatbot ChatGPT."
AI Images in Google Search Results Have Opened a Portal to Hell
https://www.404media.co/google-image-search-ai-results-have-opened-a-portal-to-hell/
"Overall, I tested this method with 13 female celebrity names, including some of the most famous people on the planet and some lesser known YouTubers and Twitch Streamers, and every single Google Image search indexed AI-generated images of them even though the search term did not include “AI.””
I Paid $365.63 to Replace 404 Media With AI
https://www.404media.co/i-paid-365-63-to-replace-404-media-with-ai/
"If that latter story sounds familiar that’s because it is a blatant, uncredited rip-off of a story I published on 404 Media the same day. I was able to publish it alongside 52 other articles that day all by myself because I created an entirely autonomous, ChatGPT-powered technology news site that steals other people’s original reporting for just $365.63.”
A Beloved Tech Blog Is Now Publishing AI Articles Under the Names of Its Old Human Staff
“Worse, the new version of TUAW has “recreated” the archives of the site by running old, real articles through a summarization tool and then republishing new, “bastardized versions” of the old articles under the bylines of real writers who didn’t actually write them”
TUAW, a site that was shut down 10 years ago, was sold to a private equity firm, then to a company in Hong Kong, and has now stolen its old workers' identities and is running their old work through AI summarizers.
How disinformation from a Russian AI spam farm ended up on top of Google search results
““In 2016, a disinformation operation like this would have likely required an army of computer trolls,” Sadeghi said. “Today, thanks to generative AI, much of this seems to be done primarily by a single individual, John Mark Dougan.””
Chum King Express
“But a former AdVon employee told The Verge that the content that AdVon says is created by humans is nearly identical to the AI-generated content they created while working there.”
AI paid for by Ads – the gpt-4o mini inflection point
https://batchmon.com/blog/ai-cheaper-than-ads/
“It's now possible to build dynamic, AI generated content entirely supported by ads. …
Will the future of the internet be entirely dynamically generated AI blogs in response to user queries? I don't know. I hope not. But the internet has been on a downward trend for a while now.”
Zombie Alt-Weeklies Are Stuffed With AI Slop About OnlyFans
https://www.wired.com/story/zombie-alt-weeklies-are-stuffed-with-ai-slop-about-onlyfans/
“Former employees are disturbed to see their archival work comingling with SEO porn slop. “It’s wrenching in so many ways,” says former Riverfront Times writer Danny Wicentowski. “Like watching a loved home get devoured by vines, or left to rot.””
Where Facebook's AI Slop Comes From
https://www.404media.co/where-facebooks-ai-slop-comes-from/
“But the videos make clear that Facebook’s AI spam problem is one that is powered and funded primarily by Facebook itself, and that most of the bizarre images we have seen over the last year are coming from Microsoft’s AI Image Creator, which is called “Bing Image Creator” in instructional videos.”
North Carolina Musician Charged With Music Streaming Fraud Aided By Artificial Intelligence
"SMITH began working with the Chief Executive Officer of an AI music company (“CC-3”) and a music promoter (“CC-4”) to create hundreds of thousands of songs using artificial intelligence that SMITH could then fraudulently stream.”
Damian Williams, the United States Attorney for the Southern District of New York, and Christie M. Curtis, the Acting Assistant Director in Charge of the New York Field Office of the Federal Bureau of Investigation (“FBI”), announced the unsealing of a three-count criminal Indictment charging MICHAEL SMITH in connection with a scheme to create hundreds of thousands of songs
Google Serving AI-Generated Images of Mushrooms Could Have 'Devastating Consequences'
https://www.404media.co/google-serves-ai-generated-images-of-mushrooms-putting-foragers-at-risk/
"The moderator, who goes by MycoMutant, noticed that when they searched for the Coprinus comatus a fungus commonly known as shaggy ink cap or shaggy mane, the first image in the Google snippet, which is featured above the search results, was an AI-generated image that looked nothing like a real Coprinus comatus."
AI Audiobook Narrators in OverDrive and the Issue of Library AI Circulation Policy
"Repeating a cover motif? Totally normal, I agree. But Seven Hundred Books? No interviews, promotional posts, or social media that I could find? That’s very strange. The Reddit “collective of authors” theory makes some sense."
Now continued at Bluesky:
https://bsky.app/profile/jtlg.bsky.social/post/3lerhqk7y2k2m
It's time to kick off the Bluesky version of a long-running Mastodon thread: how AI-generated slop on the Internet is replacing actual work made by humans for humans. I call it Gresham's Law 2.0: bad content drives out good. Here's the Mastodon thread: https://dads.cool/@[email protected]/112052300525926791 #greshasmslaw20
@jtlg 🤷
as I avoided Image/Video based social media, the impact is luckily not so big.
But yes, AI image generators allow people with two left hands (footnote: while being right-handed, no discrimination meant) to explore their imagination beyond stick figures.
@jtlg
The problem isn’t with LLM, the problem is with publishers.
Lots of marketing copy was just slop prior to LLM slop.
My apprentice had a job on the side writing advertising copy for companies. It was just drivel. He even accidentally dropped some Lorem Ipsum in once and nobody noticed!
He ended up outsourcing to India, where he paid cents in the dollar.
Finally he started outsourcing to an LLM.
His customers never proofread or checked any of the content he submitted to them.
James Grimmelmann
James Grimmelmann
Matt Blaze
Dan Morgan 
Andreas K
Dan W