Mastodawn

Cybermatron Jun 1

🔥Cassandra🔥

One of many things I'm concerned about is the growing trend of people using Gen AI in law, presenting to clients and to courts their Professional opinion, which may or may not in fact be based on existing precedent, nobody checked.

This is coming up today because there's yet another case of contempt of court: https://beige.party/@adub/115820660155470327

I noted in the comments that someone's compiling an international database where someone presented incorrect AI law in court: https://www.damiencharlotin.com/hallucinations/

In my opinion it entirely undermines the legal system, which is theoretically based on precedent, meaning specifically "what that judge said in that prior case." Not "what judges might plausibly have said, in a format that looks like fact, as programmed by tech bros."

Generative AI, I would submit, makes a mockery of the law.

#FediLaw #LawFedi

A-Dub 🥝 (@[email protected])

Using AI in court? Yikes. 😬 "In one notable case, a Toronto lawyer is facing a criminal contempt of court proceeding after including cases invented by ChatGPT in her submissions earlier this year, then denying it when questioned by the presiding judge. In a letter to the court months later, the lawyer said she misrepresented what happened out of "fear of the potential consequences & sheer embarrassment."" https://www.cbc.ca/news/canada/toronto/increasing-ai-use-canadian-courtrooms-carries-risk-9.7031131

beige.party

🔥Cassandra🔥Jan 1

In addition, a couple of weeks ago I had occasion to look something up on CanLII (https://www.canlii.org):

"a non-profit organization founded in 2001 by the Federation of Law Societies of Canada on behalf of its 14-member law societies. Its mandate is to provide efficient and open online access to judicial decisions and legislative documents. By doing so, CanLII supports members of the legal profession in the performance of their duties while providing the public with permanent open access to laws and legal decisions from all Canadian jurisdictions."

I have been, in my lifetime, a disproportionately huge fan of CanLII. Free accurate information. What a great resource! Which now uses Gen AI. https://www.canlii.org/info/generative-ai.html

I wrote to them:

"Please tell me how to turn off AI-generated captions when I'm consulting your site.

My preference would be for CanLII to turn off AI-generated captions entirely for all users, actually, because CanLII is specifically a site people go to for facts. There is currently a reported problem of lawyers presenting things in court that aren't actually legally correct because AI makes shit up. I hate that CanLII of all sources is contributing to this problem.

I hate that by using CanLII and getting AI-generated captions, I'm depleting the environment for information that *needs to be fact-checked anyway.*"

They replied:

"At the moment, there isn't an option to disable AI enrichments, but your suggestion has been shared with the appropriate team for consideration."

#FediLaw #LawFedi

Bryan Davis Jun 1

@Cassandra « AI-generated analyses should not be used to interpret source documents or in legal proceedings. » Weird that they say right on that policy page that the feature is without value. I assume that was a last ditch effort by internal objectors to address the self-inflicted problem.

🔥Cassandra🔥Jun 1

@bd808 So when someone's case gets added to the database of legal fuckups from relying on AI they can go "but we said *not* to use this service we provide."

https://www.damiencharlotin.com/hallucinations/?q=&sort_by=-date&states=Canada&period_idx=0

Bryan Davis Jun 4

@Cassandra Ah, the “water pipes intended only for use with tobacco” defense. Brilliant.

Pete Alex Harris🦡🕸️🌲/∞🪐∫Jan 1

@Cassandra
It seems like the law is aware of the mockery and is having none of it, which is a hopeful development.

It's a pity there isn't also such a thing as contempt of engineering or contempt of journalism.

@Cassandra i have a friend who works at a law firm who was gently reprimanded recently for not using copilot to do her work

🔥Cassandra🔥Jun 1

@sus Gross. What did your friend do?

@Cassandra she's trying to figure out how to use it 🤷‍♀️ . it's a requirement from the bosses.

Interpipes 💙Jun 1

@Cassandra @neil I’m going to keep saying until it starts happening: start sending these lawyers to prison for a week or so and see how fast it stops happening.

🔥Cassandra🔥Jun 1

@interpipes That is a possible outcome of contempt of court.

Interpipes 💙Jun 1

@Cassandra it sure is, but as far as I know (?), so far no court has seen fit to exact that level of punishment, and I think it clear that too few in the profession take the very well-published consequences to date very seriously

🔥Cassandra🔥Jun 1

@interpipes Yeah, someone else was just commenting that her law firm *told* her to use AI. That's a pretty significant contrast.

The most severe outcome I saw from the database of cases was monetary sanctions.

Mnemosina Jun 2

@Cassandra anglo skill issue

(Yes, it should be Cambacérès, but most anglos don't know who he is, so...)

🔥Cassandra🔥Jun 2

@Mnemosinee I don't understand this in the slightest, but I'm curious.

Mnemosina Jun 2

@Cassandra "it entirely undermines the legal system, which is theoretically based on precedent"

...not ALL legal systems.

(Cambacérès is the legal brain behind the Napoleonic code, my favourite gay legal icon. Happy Pride Month!)

🔥Cassandra🔥Jun 2

@Mnemosinee I now understand your point - I am aware of civil codes (including in Québec), which do not rely on precedent.

But, AI does seem to be infecting those legal systems too:

https://www.damiencharlotin.com/hallucinations/?q=&sort_by=-date&states=France&period_idx=0

https://www.damiencharlotin.com/hallucinations/?q=quebec&sort_by=-date&states=Canada&period_idx=0

AI Hallucination Cases Database – Damien Charlotin

The most comprehensive database of AI hallucination cases in law: legal decisions from courts worldwide, searchable by country, party, AI tool, and outcome. Updated daily.