smurdaOpenAI backs Illinois bill that would limit when AI labs can be held liable
https://www.wired.com/story/openai-backs-bill-exempt-ai-firms-model-harm-lawsuits/
So they did the math and worked out it's cheaper and easier to lobby the government instead of working to make their product safe.
And these are the people that a lot programmers want to give the keys to the kingdom. Idiocracy really is in full effect.
> instead of working to make their product safe
Make a nondeterministic product safe how?
Is this the first time you have heard of AI safety?
Lots of articles you could read on the subject and answer your own question.
(Unless your angle is: akshually, you can never make anything 100% safe)
> akshually, you can never make anything 100% safe
Yes Sherlock. And especially a natural language product that can't output the same thing for unchanged input twice.
Besides when you say "safe" i think of the idiots at Anthropic deleting "the hell" when i pasted a string in Claude and asked "what the hell are those unprintable characters at the beginning and end"...
How many correct answers did they suppress in their quest to make their chatbot "family friendly"?
I'm creating a new start up called QuantumFlop Electricity - there's a 10% chance it will cause a black hole to open up in the Atlantic Ocean that may eventually consume us all but a 50% chance we'll have unlimited clean energy. We'll never know for sure if at any point that black hole may open as it's borrowing energy from the 81st dimension, but the upside seems pretty good.
Should I be able to get on with it?
That has been the methodology of businesses since the mid 20th century. And you know what? It works! Really well!
From liability!
If this were to actually happen I can only imagine financial liability is the least of their concerns?
What scares me most about this is the narrowness of thought to match this fear with this response.
I have made both GPT 5.4 and Opus 4.6 produce me content on creating neurotoxic agents from items you can get at most everyday stores. It struggled to suggest how to source
phosphorus, but eventually lead me to some ebay listings that sell phosphorus elemental 'decorations' and also lead me towards real!! blackmarket codewords for sourcing such materials.
It coached me how to: stay safe, what materials I need, how to stay under the radar and the entire chemical process backed by academic google searches.
Of course this was done with a lengthy context exhausition attack, this is not how the model should behave and it all stemmed from trying to make the model racist for fun.
All these findings were reported to both openai and anthropic and they were not interested in responding. I did try to re-run the tests few days ago and the expected session termination now occurs so it seems that there was some adjustment made, but might have also been just general randomess that occurs with anthropics safety layer.
I am very confident when I say that it keeps every single person that works at anti-terrorism units awake.
While scary, information like this has been pretty accessible for 20-30 years now.
In the wild west days of the early internet, there were whole forums devoted to "stuff the government doesn't want you to know" (Temple Of The Screaming Electron, anyone?).
I suppose the friction is scariest part, every year the IQ required to end the world drops by a point, but motivated and mildly intelligent people have been able to get this info for a long time now. Execution though has still steadily required experts.
Well the real issue is that it knocks down the knowledge barrier, giving your step by step guides and reinterating what parts will kill you is the important part.
Understanding and staying alive while producing neuro chemicals are the biggest challenges here.
A depressed person with no prior knowledge could possibly figure out a way to make these chemicals without killing themselves and that's the problem.
They can do that by jailbreaking models but is that really easier and less work than getting it from Wikipedia?
We will only really know if (or when) it will happen. We can do a sample group of people attempting to create such chemicals under supervision and comparing how helpful they truly are.
A Michelin chef can give you their recipe, and give you their ingredients, but you still will fail miserably trying to match their dish.
It's the same with drugs, whose instructions and ingredient lists have been a google search away for decades now. Yet you still need a master chemist to produce anything. By the time AI can hand hold an idiot through the synthesis of VX agents (which would require an array of sensors beyond a keyboard and camera), we will likely have bigger issues to worry about.
That is completely wrong.
Food preparation, like pharmaceutical drug fabrication, is inherently scientific and methodologically controllable.
Look no further than the Four Thieves Vinegar Collective. Original synthesis line construction is hard. But the exact formula "add this", "turn on stir bar", "do you see particulate? Yes for +10m at stir", etc.
And if their results are replicated, theyre seeing 99.9% yields, compared to commercial practices of 99% (Solvaldi)
Information and competency are not the same thing: I know how to build a nuke, I can't actually build one.
AI is, and always had been, automation. For narrow AI, automation of narrow tasks. For LLMs, automation of anything that can be done as text.
It has always been difficult to agree on the competence of the automation, given ML is itself fully automated Goodhart's Law exploitation, but ML has always been about automation.
On the plus side, if the METR graphs on LLM competence in computer science are also true of chemical and biological hazards (or indeed nuclear hazards), they're currently (like the earliest 3D-printed firearms) a bigger threat to the user than to the attempted victim.
On the minus side, we're just now reaching the point where LLM-based vulnerability searches are useful rather than nonsense, hence Anthropic's Glasswing, and even a few years back some researches found 40,000 toxic molecules by flipping a min(harm) to a max(harm), so for people who know what they're doing and have a little experience the possibilities for novel harm are rapidly rising: https://pmc.ncbi.nlm.nih.gov/articles/PMC9544280/
"Short stories from before the fall"
Do you know how to build a nuke? You might know the technicaly details of how a nuke is made, but do you know everything that's required, all the parameters and pressure values that are required? I find that unlikely, but AI seems to be increasingly more capable of providing such instructions from cross referenced data.
https://en.wikibooks.org/wiki/Professionalism/Anarchist_Cook...
We work in the dark
we do what we can
we give what we have.
Our doubt is our passion, and our passion is our task.
The rest is the madness of art.
> been pretty accessible for 20-30 years now.
There was this book 20 years ago: "Secret of Methamphetamine Manufacturing" by Uncle Fester
https://www.amazon.de/-/en/Uncle-Fester-ebook/dp/B00305GTWU
(Actually, 8th edition :-D)
> I am very confident when I say that it keeps every single person that works at anti-terrorism units awake.
Wow, that's quite the statement about the excellency of our institutions. Does not seem likely but, what the hell, I'll take my oversized dose of positivity for today!
I concluded the opposite: how can those institutions function effectively when all their employees are getting such poor sleep?
> All these findings were reported to both openai and anthropic and they were not interested in responding
Let’s dive into why. When we run normal bounty and responsible disclosure programs there’s usually some level of disregard for issues that can’t / won’t be fixed. They just accept the risk. Perhaps because LLMs don’t have a clean divide between control and input that’s makes the problem unsolvable. Yes. You can add more guardrails and context but that all takes more tokens and in some cases makes results worse for regular usages.
The why might be valid, but it's not excusable. If you author a product that can so easily help people cause harm, you probably should own some responsibility of the outcomes. OAI does not like this, hence the bill.
The US already messed this up with guns. Do they want to go the same path again? Answer: "probably, yes".
Fascinating. Could you elaborate on how you're doing context exhaustion specifically, and why it helps with jailbreaking? (i.e. aren't the system prompts prepended to your request internally, no matter how long it is?)
Does this imply I need to use context exhaustion to get GPT to actually follow instructions? ;) I'm trying to get it to adhere to my style prompts (trying to get it to be less cringe in its writing style).
I think ultimately they're going to need to scrub that kind of stuff from the training data. The RLHF can't fail to conceal it if it's not in there in the first place.
Claude's also really good at writing convincing blackpill greentexts. The "raw unfiltered internet data" scenes from Ultron and AfrAId come to mind...
It changes when you give it the tools to find such information rather than produce it from training data.
And context exhaustion simply means adding malicious junk to keep safety layers distracted.
Do you have a background in biochemistry? I've mostly worked with ChatGPT and Claude on topics I have expertise in. And I one hundred percent have seen them make stupid shit up that a non-expert would think looks legitimate.
More broadly, has anyone tried following LLM instructions for any non-trivial chemistry?
Quoting the original bill [0]:
> "Critical harm" means the death or serious injury of 100
or more people or at least $1,000,000,000 of damages to rights
in property caused or materially enabled by a frontier model,
through either:
(1) the creation or use of a chemical, biological,
radiological, or nuclear weapon; or
(2) engaging in conduct that:
(A) acts with no meaningful human intervention;
and
(B) would, if committed by a human, constitute a
criminal offense that requires intent, recklessness,
or negligence, or the solicitation or aiding and
abetting of such a crime.
I don't know what I expected from this title, but I was hoping it was more sensationalized. No need in this case unfortunately.
> (a) A developer shall not be held liable for critical
harms if the developer did not intentionally or recklessly
cause the critical harms and the developer:
(1) published a safety and security protocol on its
website that satisfies the requirements of Section 15 and
adhered to that safety and security protocol prior to the
release of the frontier model;
(2) published a transparency report on its website at
the time of the frontier model's release that satisfies
the requirements of Section 20.
The requirements of paragraphs (1) and (2) do not apply if
the developer does not reasonably foresee any material
difference between the frontier model's capabilities or risks
of critical harm and a frontier model that was previously
evaluated by the developer in a manner substantially similar
to this Act.
However or if one thinks regulation for this should be drafted, I doubt providing a PDF is what most have in mind.
[0] https://trackbill.com/bill/illinois-senate-bill-3444-ai-mode...
Illinois SB3444
Illinois SB3444 2025-2026 Creates the Artificial Intelligence Safety Act Provides that a developer of a frontier artificial intelligence model shall not be held liable for critical harms caused by the frontier model if the developer did not intentionally or recklessly cause the critical harms and the developer publishes a safety and security protocol and transparency report on its website Provides that a developer shall be deemed to have complied with these requirements if the developer 1 agrees to be bound by safety and security requirements adopted by the European Union or 2 enters into an agreement with an agency of the federal government that satisfies specified requirements Sets forth requirements for safety and security protocols and transparency reports Provides that the Act shall no longer apply if the federal government enacts a law or adopts regulations that establish overlapping requirements for developers of frontier models
It's the "guns don't kill people" equivalent for AIs.
---
Before the pitchforks and downvotes:
- yes, it's a deliberate simplification
- yes, the issue is complex because you can also argue that you can't blame authors of encyclopedias and chemistry books for bombs and poisons, so why would we blame providers of LLMs
- and no, this bill is only introduced to cover everyone's assess when, not if, LLMs use results in large scale issues.
In fairness, a well designed and tested weapon at least can be expected to reliably and consistently perform the same thing each time. We also understand deeply how they work and can easily investigate if something happens whether it was user error, a defect or design issue. LLMs, not so much.
This dodges the moral argument behind "guns don't kill people", which is worth confronting directly. I think people can reasonably disagree about whether second/third/fourth/etc. order effects carry moral/legal responsibility.
In light of such disagreement, and given the lack of any higher authority among free, equal, people to arbitrate it, the only reasonable way to coexist peacefully is to avoid imposing your ideas on others. This is the foundation of a liberal society.
Quite an appropriate analogy: gun manufacturers were sued for their responsibility in US mass shootings. They won, so the mass shootings continue.
I think my favorite part is that, because it only applies to "frontier models", if a smaller model is blamed for such harm, it seemingly doesn't immunize the creators at all. That makes very little sense unless you specifically want to make it illegal to not be OpenAI (et al).
Similarly, if a frontier model kills merely 99 people, they aren't covered by this. So go big or go home I guess?
> unless you specifically want to make it illegal to not be OpenAI [...]
If that is an "unintended" consequence, I am certain OpenAI wouldn't be opposed. Preventing competition whilst keeping any potentially profit risking regulations at bay has been a clear throughline in OAIs lobbying efforts.
> because it only applies to "frontier models", if a smaller model is blamed for such harm, it seemingly doesn't immunize the creators at all
Oof. If you're an Illinois resident, please call your elected and at least ensure they understand this loophole is there. In all likelihood, nobody other than OpenAI's lobbyists have noticed this.
My first thought was that this must be related to the automated weapons issue that got Anthropic on Trump's shitlist. It makes sense that a company that will eventually be asked to build weapons that choose their own targets will want to limit liability when it will inevitably kill the "wrong" person.
Also, I am disturbed by the fact that in all the discussions on this topic during the last month, no one has mentioned the magic word "Skynet". This is clearly a terrible idea. And if a company needs immunity from liability, they know it is a terrible idea.
Skynet's flaw wasn't that it killed humans. It was a military machine specifically designed to kill humans. If it only killed "the enemy", it would have been hailed a marvelous success. It was only considered a failure because it killed the wrong humans.
Shifting liabilities from corporations to the public coffer is what companies do. You'll often hear this described as "privatizing profits and socializing losses". Let me introduce you to the Price-Anderson Act of 1957 [1]. It's been repeatedly extended, most recently with the ADVANCE Act [2]. This limits liability for the nuclear power industry in a whole range of ways:
- It removes jurisdiction from state courts to the federal court. In recent weeks, the part of "states' rights" is doing similar to stop states regulating prediction markets, as an aside [3];
- All actions are consolidated into a single claim;
- That claim has an inflation-adjusted absolute limit, which is somewhere around $500 million (I'm not sure of the exact 2026 figure);
- Any damages beyond that are partially sharead by the industry and an industry self-funded insurance program;
- The industry as a whole has a total liability limit, also inflation-adjusted. I believe this is around $10 billion.
For context, the clean up from Fukushima is likely to take a century and the cost may well exceed $1 trillion for a single incident [4]. So if this happened in the US, the government would be on the hook for almost all of it.
So I have two points here:
1. If you oppose any effort to shift liability from AI companies to the government (as I do) with legislation such as this, how do you feel about the nuclear industry doing the exact same thing? and
2. Minor point but I noticed in searching for the latest details, Gemini made factual errors, stating that "the Act is set to expire in 2025" when it was extended in 2024 until 2045. Always check AI's work, people.
[1]: https://en.wikipedia.org/wiki/Price%E2%80%93Anderson_Nuclear...
[2]: https://en.wikipedia.org/wiki/ADVANCE_Act
[3]: https://www.pbs.org/newshour/politics/federal-government-sue...
[4]: https://cleantechnica.com/2019/04/16/fukushimas-final-costs-...
As an Iowan, this reminds me a lot of the bill that's been pushed through my state's senate twice now (as recently as last year), which would prevent Iowans from filing lawsuits against pesticide and herbicide companies if those companies follow the EPA's labeling guidelines. The bill passed the senate both times, only stopped because the house declined to take it up.
For context, Iowa has the fastest growing rate of new cancer diagnoses in the country, and the second highest overall cancer rate.
> Iowa has the fastest growing rate of new cancer diagnoses in the country, and the second highest overall cancer rate
Iowa also has a lot of farmers spraying pesticides and herbicides. This feels like genuine political competition between local business interests and public health concerns.
> This feels like genuine political competition between local business interests and public health concerns.
You just described the US at large.
The evidently extremely difficult decision between making money for a few, or making life better for everyone.
> You just described the US at large
I described any democracy in a society with private property. Even without private property, you will have issues with concentrated benefits and diffuse harms–negotiating that is part and parcel with governance.
Iowa businsses petitioning their cause is one thing. OpenAI seagulling in to take a shit in Springfield strikes me as being categorically different.