A court in Munich declared that Google is liable for their "AI summaries" and all its hallucinations. This is an important step to bring "AI" slop in line with all other products on the market: "AI" products are basically the only ones where a provider can just deliver unchecked garbage and put all the liability on the consumer. I hope to see aggressive change here.

https://the-decoder.com/landmark-german-ruling-declares-googles-ai-overviews-are-googles-own-words-and-makes-it-liable-for-false-answers/

Landmark German ruling declares Google's AI Overviews are Google's own words and makes it liable for false answers

A German regional court has ruled that Google is directly liable for the content of its AI search overviews. According to the court, previous limited liability protections for search engine operators don't apply to AI overviews. In this case, Google's AI had falsely linked two publishers to fraud and made claims that didn't appear in any of the linked sources. The ruling could set a precedent for AI-generated content liability worldwide.

The Decoder
@tante wow, I hope this holds up.
@tante also, how would the "do your fact checking yourself"-approach work at all once the plan to eliminate links is enacted? Maybe making your bullshit all that transparent isn't the best move.
@tante
I also hope they don't say "chatbots can make mistakes", because epistemologically, they cannot. They do not pursue trueness.
@tante
They could flag it as comedy or satire
@iamlayer8 @tante "Google AI Overview is for Entertainment Purposes Only" just like ... was it copilot that had this in the terms of use or whatever?

@timotimo
It should be flagged as AI Slop - on screen, mandated by law.
Then they can re-define what that means by delivering correct information. And in 5 years we will use #AISlop as an indicator of high quality content.

@tante

@tante This is getting really interesting. First the chatbots and now the "Ai search results". The faulty design starts to show and become useless if they can get sued for every misinformation. It will be easier to train humans for this jobs, at least you can yell at them when the fuck up :-)

@tante ”A German court has ruled that Google is directly liable for what its AI search overviews say. Previous case law shielding search engine operators from liability doesn't apply to AI overviews.”

This is the correct action to take. You are operating a technology that produces this information. You are not connecting people to somebody else’s voice.

@gimulnautti @tante This should be cemented on the EU level. It might get Google thinking about spreading lies.

@tante

Just wanted to look for my SPIEGEL pictures of Nazi Klar, pissed at a demo.
It was a short while ago after he was caught.

google answered that I am not a RAF terrorist.

Do I get compensation?

[de]

@sl007 "helpful"

@tante

yeah, "helpful" was a month before when it showed me the image I searched for.

This search returned 0 results and 3 wrong connections ;)

@tante This is the kind of thing I couldn't have come up with, because I would never have considered what the LLM is spitting out to be Google's words.

Guess there is no real way out. They get what they wanted and I isn't legally theft (however incompetent)... but now it's their words so guess they are responsible.

Look forward to how they try to fight this one. Sorry it's not our words, we actually stole the entirety of human creation.

@theeclecticdyslexic @tante they made/own/run the thing, so why not? They've put the summaries front and centre. They integrated it when there was absolutely no need. The LLM didn't have to be doing any of those things.

@noodlemaz @tante Oh, for sure. I just never would have thought of it, because I was so preoccupied with the legal questions of theft, and how anything it says is almost always in the training data somewhere (or multiple somewheres).

I do think this ruling is a consequence of their efforts to convince people it is "intelligent". People wouldn't consider it something to listen to otherwise. It would just be an inconvenience, as it is for people who know better.

@tante

Google's defence needs to be amplified by anyone talking to politicians about 'AI' regulation:

Google is explicitly saying in their legal filing that the outputs from their LLM should not be trusted and that users should know that.

That's one hell of an admission. Imagine saying that about any other category of product.

@david_chisnall @tante

iirc, Google said that earlier about their search results, also.

@lobingera
Slightly different. "search results you see may not be the most relevant for your query" is very different to "We're providing actual information which has a reasonably likelihood of being wrong, and potentially dangerously wrong".
@david_chisnall @tante
@markotway @lobingera @david_chisnall @tante Another big difference here is that with search results the original site is "responsible" for the information. With AI the information is provided by Google.
@david_chisnall @tante this is also known as the Fox “News” defense for those in the US
@david_chisnall
Imagine TI saying that about their calculators.
@tante

@ozzelot
I remember Microsoft successfully lobbying to keep known faulty behavior like handling the year 1900 as a leap year in the OOXML standard.

So that idea is not that far off for me.

@david_chisnall @tante

@david_chisnall @tante

What's the difference between an LLM and a pack of cigarettes?

They both fill the room with smoke that will eventually kill someone. But cigarettes come with a warning.

@david_chisnall @tante Remember when Intel released a chip with a built in floating point math error that happened one nine-billionth of the time?

https://en.wikipedia.org/wiki/Pentium_FDIV_bug

Pentium FDIV bug - Wikipedia

@david_chisnall @tante Don't forget that Copilot is for entertainment purposes only, too.

@david_chisnall

@tante

I know a few people who won´t like this admission from Google: "the outputs from their LLM should not be trusted and users should know that" !!

Will OpenAI e.a. admit the same thing?

@david_chisnall @tante

Indeed no other industry or product could get away with that ... - let us just hope that this court's view is upheld through future instances

@sebastian @david_chisnall @tante

I think the odds of that are zero, I'm assuming that Google will find someone to bribe before long.

@sebastian @david_chisnall @tante

also,
Note that this is not a final court ruling, just a so called "preliminary injunction" (einstweilige Verfügung) by a relatively low-ranking court -
there are unfortunately many ways in which this could be overturned.

#Google #AI #court #germany

Court ruling:

https://the-decoder.de/wp-content/uploads/2026/06/26_O_869_26_begl_Abschrift_Urteil_v_28_05_2026_Geschwaerzt_Geschwaerzt_Geschwaerzt.pdf

context, german:

https://www.golem.de/news/google-stoererhaftung-gilt-nicht-fuer-ki-uebersicht-von-suchergebnissen-2606-209625.html

@david_chisnall Don't most software licences, particularly FOSS ones, say that?

@tante

@edavies @tante

No, they say that the product comes with no liabilities. That limitation of liability is not absolute and is restricted to the degree to which laws allow liability to be disclaimed. If you put something actively malicious in an open-source project, that license doesn't absolve you.

If you make explicit claims about the product, then a license saying 'actually, does not do the things that we claimed it can do' will not protect you against fraud claims.

@david_chisnall @tante I see nothing shocking in that admission. AI abilities are jagged - in some areas exceptional (radiology) and in other prone to error. Like any human.

The key to judging its veracity is based on the strength of the citations it provides and what other sources are saying about the questions we ask of it.

@lymphomation @david_chisnall @tante
AI abilities are "jagged" because AI is an umbrella term for a bunch of disparate technologies and the AI used for radiology is almost entirely unrelated to the AI at issue here

#GenAI

@lymphomation @david_chisnall @tante
As for judging the veracity of LLM output based on its citations and what other sources say, by the time you've done that, you would've done better not to use an LLM at all - it would've been both less work and higher quality output

#GenAI

@sabik @lymphomation @tante

It's also not good. It turns out that existing ML models trained on x-ray data overfit on specific measurement errors for individual x-ray machines and produce surprisingly poor results when you try to use them on a different x-ray machine, of the same model in the same hospital, let alone a different model.

There was a paper published near the start of the year debunking a load of the claims about ML in radiology.

But that doesn't stop it being the go-to example for boosters.

@david_chisnall Could you, please, link the paper?

@david_chisnall @sabik @tante

I base my perspective on review articles published to journals. This is one of them with an image capture of the review articles conclusion. This one of many such published papers.

Review
Redefining Radiology: A Review of Artificial Intelligence
Integration in Medical Imaging

@sabik @lymphomation @david_chisnall @tante

"But Science!" is just the new talking point for AI astroturf.

Notice how all the boosters started saying that and "jagged" at the same time?

@david_chisnall @tante that's the same defense Nigel Farage uses "I wasn't inciting a riot, I was just asking questions!". Although it seems by now he is so confident in the right-wing media barons who promote him that he even dispensed with the pretense of plausible deniability.
@david_chisnall @tante bc on linux says "without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE." which my old school calculator (from 1970s that still works) does not...

@tforcworc @tante

As I've said elsewhere today:

There are strict legal limits on where you can limit liability. Your calculator can't have that disclaimer at all because (in both the EU and USA) there are very strict limits on disclaimers of liability for physical machines (which is an issue that comes up in open-source hardware quite often).

Even in software, claiming in your marketing that your product does one thing and then having a disclaimer in the license that says that it does not, in fact, do that thing is generally a problem: you may not be liable for the damages from failing to do the thing, but there's a good chance that you're liable for fraud. A disclaimer of liability isn't a get out of jail free card, it's a statement of intent.

@david_chisnall @tante once more, the law makes arbitrary distinctions. What if I grow a biological calculator? (which one can...)

@tforcworc @tante

It's not really an arbitrary distinction. The relevant law treats software as a component. The liabilities apply to a final product. The product liability laws cover products delivered to customers. It's the responsibility of the product builder to ensure that components meet the requirements and to use contract law to enforce any liability that's necessary to propagate along the supply chain.

The EU's CRA takes a similar view: an open-source project does not have any liability but a product that incorporates that project must do its own due diligence to ensure compliance.

@david_chisnall @tante the law depends on its interpretation. see for example, this view on liability and machine learning https://papers.ssrn.com/sol3/papers.cfm?abstract_id=2853462
@david_chisnall @tante Completely on-brand for the software industry, famous for "no promises, no warranties" EULA bullshit.

@david_chisnall @tante

Q-tips says it. "don't clean your ears". What do 90% of Q-tips buyers do? Clean their ears.

@nosrednayduj @david_chisnall @tante I can tell I haven't had enough caffeine yet, because now my brain is going down the rabbit hole of "we have to clean our ears with Q-tips because getting it done safely at the doctor is not financially accessible to most Americans."

But Europeans probably do it too, and I know Asian countries have those little ear spoons which presumably have similar risks of eardrum puncture.

@meredith @nosrednayduj @david_chisnall @tante But East Asian earwax is different from European and African earwax, so the tools and techniques are different. https://www.mcgill.ca/oss/article/did-you-know-general-science/your-earwax-and-sweat-have-something-smelly-common
Your Earwax and Sweat Have Something Smelly in Common

Did you know that there are two types of earwax and that, depending on where you are from, you have one or the other? Earwax, also known as cerumen, is produced by glands in the ear canal. It’s actually a mix of secretions from two different types of glands to which skin flakes, dust and debris get added. Its function is debated, but it probably acts to trap bacteria and insects and to keep the ear canal lubricated. Earwax moves out of the ear according to a conveyer belt mechanism: skin cells inside the ear transport it out by migrating out of the ear at a rate of 1/20th of a millimetre every day, similar to the speed at which our nails grow. The vast majority of people of European or African descent have wet earwax. Like honey, it is yellow and sticky. Most East Asian people, however, have dry earwax, which is grey, brittle, and non-sticky. The landscape becomes much more muddied, however, when looking at people from the Pacific Islands, South and Central Asia, as well as Indigenous populations of North America, where 30 to 50% have dry earwax types. By looking at families, we’ve known for a while that wet earwax was inherited in a dominant fashion. We receive two copies of most genes, one from our mother and one from our father. A dominant trait like wet earwax means that as long as a person has one copy of the wet earwax gene, they will produce wet earwax. They could have received a copy from their mother or from their father or from both: their cerumen will be yellow and sticky. To make dry earwax, they would need to receive the dry earwax version of the gene from both their mother and their father. The gene that determines the type of earwax the body produces was identified in 2006 after a big clue to its location had been observed four years prior. A Japanese woman presented with a rare disorder, whose name will delight fans of Scrabble: paroxysmal kinesigenic choreoathetosis. It means she had brief episodes of involuntary contractions and twisting and writhing of her arms and legs. Six other people in her family also had these symptoms, which suggested a genetic origin. That, in and of itself, is unremarkable. The curious thing was that this rarity was not limited to the choreoathetosis: they also all had wet earwax, which is very uncommon among Japanese people. Two rare genetic traits affecting the same people in a family? It probably meant that both genes were close together on the same chromosome, because the frequent DNA recombinations that occur during the making of sperm and eggs would have separated them otherwise. A team of Japanese scientists used this knowledge to map the earwax gene to chromosome 16 and a few years later they found the exact gene that was responsible for the consistency and colour of someone’s earwax. In a world in which genetics can be extremely complicated, what the researchers found was the simplest possible explanation: a one-base change—a single-letter change in the A, C, T, G alphabet of the DNA molecule—was responsible. People who had a G at a specific spot in this gene produced wet earwax, and those with an A on both copies of the gene made dry earwax. It was that easy. The more interesting question is why. DNA makes proteins, and this particular stretch of DNA, a gene called ABCC11, codes for a protein called a transporter. In people with wet earwax, this transporter protein can be found inside the glands that produce some of the secretions present in earwax, and it helps with the release of these secretions. But in people with dry earwax, the protein cannot bind to a particular sugar. It is seen as defective and the cell breaks it down before it can fulfill its role. Seen through this molecular lens, you may think that people who have dry earwax are “defective” in some way but that is not true. Dry earwax in itself does not seem to create any problem. In fact, this one-letter change in the DNA code for the ABCC11 gene has a pleasant consequence: less stinky sweat. Indeed, some of the same glands involved in producing earwax secrete sweat elsewhere in the body. People with wet earwax not only release smelly compounds in their cerumen thanks to the ABCC11 transporter protein: they also release smelly compounds through their armpit sweat, a trait known to Scrabble players everywhere as axillary osmidrosis. The mechanism is simple. In both wet earwax and its corresponding armpit sweat, ABCC11 transports certain natural molecules out of cells, and bacteria on the skin feast on them, transforming them into smelly, volatile organic compounds. Scientists have used gas chromatography/mass spectrometry to identify these compounds in both wet and dry earwax and, sure enough, they found larger quantities of them in the earwax of white volunteers compared to that of East Asian participants. Earwax may be gross but the scientific discoveries surrounding it could lead to diagnostic applications. A molecule called sotolone causes the sweet smell found in maple syrup urine disease, in which some of the building blocks of proteins found in food can’t be broken down, and sotolone can be detected in the earwax of infants who have this condition. The SARS-CoV-2 coronavirus can also sometimes be detected in earwax samples from patients with COVID-19. But more interestingly, a problem with the inner ear called Ménière’s disease, in which people experience vertigo and hearing loss, is typically diagnosed by excluding everything else; but a team in Albany, New York recently reported that the earwax of these patients has lower levels of three fatty acids. Interrogating cerumen samples with a mass spectrometer may one day help doctors more quickly diagnose patients with this condition. As for what to do with earwax, the answer is to simply leave it alone until it makes its way out of the ear canal. Forcing a Q-tip down the canal can actually push earwax further down and compact it, which can affect hearing. It can even tear through the eardrum. Unless it is causing problems, we should simply let the earwax be, whether it is wet or dry. Take-home message: - Earwax can either be wet or dry, which is decided by a single letter change in a gene called ABCC11 - People of European or African descent tend to have wet earwax, whereas most East Asians have dry earwax - Because the protein made by ABCC11 is responsible for secreting molecules that are metabolized by bacteria into smelly compounds, people with wet earwax also produce armpit sweat with a more pronounced smell @CrackedScience

Office for Science and Society
@PeterLudemann @nosrednayduj @david_chisnall @tante Right, which is why I mentioned the different tool and the danger of eardrum puncture vs the added Q-tip danger of leaving the fibers in the ears.

@david_chisnall @tante I remember years ago being on the Tube in London, and the it stopped, and the recommendation was "Please use other means to get to your destination". Which I felt was rather like saying "We cannot do our job, please find someone who can do our job of getting you there"

It feels a little like this.

@SteveClough @david_chisnall @tante I was once in Tokyo when there was a stoppage on a metro line and they were handling out tickets for other lines and apologizing profusely. And in both Japan and Canada, saying "I'm sorry" is not an admission of liability.

@david_chisnall @tante I would love for someone to hold them to actually proving that statement.

Do people?

Does the general public, in fact, treat LLM output as just statistically likely text, not real information?

All of their advertisement sure seems to be designed to ensure people *don’t*, and instead put all their trust in this machine garbage.

@david_chisnall
It’s fascinating what truths people suddenly find within themselves when lying has negative consequences. ;)

@tante

@david_chisnall @tante can you please point me to the exact wording? I need it for a workshop I'm preparing, thanks!

@tante

Excellent verdict, actions have consequences…