surprisetalkOpenAI says its new model GPT-2 is too dangerous to release (2019)
https://slate.com/technology/2019/02/openai-gpt2-text-generating-algorithm-ai-dangerous.html
Had a minor conniption until I saw the year. OpenAI just struggled to close a round. And the New Yorker just published an unflattering profile of Altman [1]. So it would make sense they'd go back to the PR strategy of "stop me from shooting grandma."
[1] https://www.newyorker.com/magazine/2026/04/13/sam-altman-may...
I didn’t notice the year in the headline. My poor meemaw still had a few months of clarity ahead of her, but this headline stole it away from her
I think they are right unintentionally. The growing amount of low-quality content everywhere could become a real problem.
Now imagine all that low quality AI slop is being posted online and a new generation of AI will "learn" from it, output it's own version of AI slop, that will eventually end up online again for a new generation of AI to "learn" from.
Something, something, idiocracy comes to mind.
> Something, something, idiocracy comes to mind.
So, confirmation? They are catching up quickly!
The actuality is, anyone with pre-slop data still has their pre-slop data. And there are endless ways to get more value out of good data.
Bootstrapping better performance by using existing models to down select data for higher density/median quality, or leverage recognizable lower quality data to reinforce doing better. Models critiquing each other, so the baseline AI behavior increases, and in the process, they also create better training data. And a thousand more ways.
Managed intelligently, intelligence wants to compound.
The difference between human and AI idiocracy, is we don't delete our idiots. I am not suggesting we do that. But maybe we shouldn't elect them. Either way, that is one more very steep disadvantage for us.
The AI centipede
This leads to a well-documented phenomenon known as model collapse. You know how if you blur and sharpen an image repeatedly you eventually end up with just a rectangle of creepy, wormy spaghetti lines? You lose information on each blur, and then ask it to reconstitute the image with less information on each sharpen, until there's nothing recognizable left.
Training a model is like the blur and generating from that model is like the sharpen. Repeat enough times and enough information is lost that you're just left with "wormy spaghetti lines"—in an LLM's case, meaningless gibberish that actually pretty closely resembles the glitchy stuff said by the cores that fall off GLaDOS in Portal. I dunno, you read the paper and be the judge:
https://www.nature.com/articles/s41586-024-07566-y
To jump to the last output sample, C-f Gen 9
Of course you may be talking about the human aspect of this. Gods willing, we'll realize that our LLMs are spewing gibberish and think twice about putting them in all the things, all the time. But the scenario I fear isn't Idiocracy—it's worse: a community of humans who treat the gibberish as sacred writ, Zardoz style.
AI models collapse when trained on recursively generated data - Nature
Analysis shows that indiscriminately training generative artificial intelligence on real and generated content, usually done by scraping data from the Internet, can lead to a collapse in the ability of the models to generate diverse high-quality output.
They were more than right. They were correct in an intentional, precise manner. This is what OpenAI actually stated[0]:
> Synthetic imagery, audio, and video, imply that technologies are reducing the cost of generating fake content and waging disinformation campaigns.
> ‘The public at large will need to become more sceptical of text they find online, just as the ”deep fakes” phenomenon calls for more scepticism about images.
It ended up just like that.
[0]: https://metro.co.uk/2019/02/15/elon-musks-openai-builds-arti...
Yeah, I find it a bit odd how at the time everyone was pointing and laughing at OpenAI for being obviously wrong about this. Now in 2026, AI slop is very obviously a serious problem - it inundates all platforms and obscures the truth. And people are still saying OpenAI in 2019 were wrong?
Both crowds are right because two messages were spread. The researchers spread reasonable fears and concerns. The marketing charlatans like Altman oversold the scare as "Terminator in T-4 days" to imply greater capacity in those systems than was reasonably there.
The problem is the most publicly disseminated messaging around the topic were the fear mongering "it's god in a box" style messaging around it. Can't argue with the billions secured in funding heisted via pyramid scheme for the current GPU bonfire, but people are right to ridicule, while also right to point out warnings were reasonable. Both are true, it depends on which face of "OpenAI" we're talking about, researchers or marketing chuds?
Ultimately AGI isn't something anyone with serious skill/experience in the field expects of a transformer architecture, even if scaled to a planet sized system. It is an architecture which simply lacks the required inductive bias. Anyone who claims otherwise is a liar or a charlatan.
The fact that they knew they were shitting in the public well and did it anyways pisses me off. What colossally selfish assholes.
Hang them all.
It already is a real problem. Large portions of the internet are completely untrustworthy because of the sheer slop volume.
Lol. The vast majority of content has always been low-quality. Those who believe that things were better before LLMs have selective memory.
The quality hasn't changed. The volume has. It used to take real human time to create garbage. There was value in that. Someone though "Hmm, what worthless thing can I do? I know! I'll make people online mad." And then they spent the time getting someone else's goat. It was great. A good balance, spreading lies took some minimum effort. Now we have automated garbage. And the flavor of the garbage is: gaslighting people with an illusion of community. We've empowered the trolls with an infinite meme-o-rater while ignoring the real human time spent unwillingly sifting through the ever increasing pile of worthlessness.
The world does not have to get worse. We're letting it though.
Maybe that's true, But I think before LLMs became common, people had more distinct ways of expressing themselves, low-quality for not. Now, a lot of online writing feels uniform and I think that is worse.
The current "too dangerous" hype today is Anthropic's Mythos. They say it is so mighty that they will wall it off and only grant access to approved corporations.
Ah yes, corporations, famously the right hands to wield mighty weapons.
Someone needs to make a compilation of all these classic OpenAI moments. Including hits like GPT-2 too dangerous, the 64x64 image model DALL-E too scary, "push the veil of ignorance back", AGI achieved internally, Q*/strawberry is able to solve math and is making OpenAI researchers panic, etc. etc.
I use Codex btw, and I really love it. But some of these companies have been so overhyping the capabilities of these models for years now that it's both funny to look back and tiresome to still keep hearing it.
Meanwhile I am at wits end after NONE OF Codex GPT-5.4 on Extra High, Claude Opus 4.6-1M on Max, Opus 4.6 on Max, and Gemini 3.1 Pro on High have been able to solve a very straightforward and basic UI bug I'm facing. To the point where, after wasting a day on this, I am now just going to go through the (single file) of code and just fix it myself.
Update: some 20 minutes later, I have fixed the bug. Despite not knowing this particular programming language or framework.
> I am now just going to go through the (single file) of code and just fix it myself.
That's front page news, in this era.
Thank you for the belly laugh.
I understand how laughable that sounds when I say it out loud. But the reality is, when I'm in a state of 'Tell LLM what to do, verify, repeat', it's really hard to sometimes break out of that loop and do manual fixes.
Maybe the brain has some advanced optimization where once you're in a loop, roughly staying inside that loop has a lower impedance than starting one. Maybe that's why the flow state feels so magical, it's when resistance is at its lowest. Maybe I need sleep.
> Maybe the brain
…is already damaged by reliance on AI.
> it's really hard to sometimes break out of that loop and do manual fixes
You're aware of the MIT Media Lab study[0] from last summer regarding LLM usage and eroding critical thinking skills...?
[0] Your Brain on ChatGPT: Accumulation of Cognitive Debt when Using an AI Assistant for Essay Writing Task
June 2025
DOI:10.48550/arXiv.2506.08872
>> it's really hard to sometimes break out of that loop and do manual fixes
it's not just an erosion of skills, it can also break the whole LLM toolchain flow.
Easy example: Put together some fairly complicated multi-facet program with an LLM. You'll eventually hit a bug that it needs to be coaxed into fixing. In the middle of this bug-fixing conversation go and ahead and fire an editor up and flip a true/false or change a value.
Half the time it'll go un-noticed. The other half of the time the LLM will do a git diff and see those values changed. It will then proceed to go on a tangent auditing code for specific methods or reasons that would have autonomously flipped those values.
This creates a behavior where you not only have to flip the value, the next prompt to the LLM has to be "I just flipped Y value.." in order to prevent the tangent that it (quite rightfully in most cases) goes off on when it sees a mysteriously changed value.
so you either lean in and tell the llm "flip this value", or you flip the value yourself and then explain. It takes more tokens to explain, in most cases, so you generally eat the time and let the LLM sort it.
so yeah, skill erosion, but it's also just a point of technical friction right now that'll improve.
Are you sure they are not just refusing to solve your UI bug due to safety concerns? They may be worried you'll take over the world once your UX becomes too good.
> a very straightforward and basic UI bug
Show us the code, or an obfuscated snippet. A common challenge with coding-agent related posts is that the described experiences have no associated context, and readers have no way of knowing whether it's the model, the task, the company or even the developer.
Nobody learns anything without context, including the poster.
That's hard to believe in my case. I tried a variety of prompts, 3 different frontier models, provided manual screenshot(s), the agent itself also took its own screenshots from tests during the course of debugging. Nothing worked. I have now fixed the bug manually after 15-20 minutes of playing around with a codebase where I don't know the language and didn't write a single line of code until now.
I had a problem that required a recursive solution and Opus4.6 nearly used all my credits trying to solve it to no avail. In the AI apocalypse I hope I'm not judged too harshly for my words near the end of all those sessions lol.
This is obviously in response to Mythos, but I'll actually defend their statement at that time - they were right to take a pause.
Think about how much things have changed in our industry since GPT-2 has dropped - it WAS that dangerous, not in itself, but because it was the first that really signaled a change in the field of play. GPT-2 was where the capabilities of these were really proven, up until that point it was a neat research project.
Mythos is similar. It's showing things we haven't seen before. I read the full 250 page whitepaper today (joys of being pseudo-retired, had the hours to do it), and I was blown away. It's capabilities for hacking are unparalleled, but more importantly they've shown that they've made significant improvements in safety for this model just in the last month, and taking more time to make sure it doesn't negatively affect society is a net positive.
Shut up