Our dramatis personae - some lawyers in federal court, in a lawsuit over a personal injury on an airplane.
Bartholomew Banino (BB) represents the airline.
Peter LoDuca (PL) and Steven Schwartz (SS) represent the injured person.
Here's the docket (via @questauthority) https://www.courtlistener.com/docket/63107798/mata-v-avianca-inc/?filed_after=&filed_before=&entry_gte=&entry_lte=&order_by=asc
On March 1st, PL files that reply/opposition to the motion to dismiss.
It cites a bunch of cases in support of its argument! They seem quite convincing - there's some state courts holding that state courts can decide international airline accidents. (See pages 4 and 5). https://storage.courtlistener.com/recap/gov.uscourts.nysd.575368/gov.uscourts.nysd.575368.21.0.pdf
REPLY MEMORANDUM OF LAW in Support re: 16 MOTION to Dismiss pursuant to Fed. R. Civ. P. 12(b)(6). . Document filed by Avianca, Inc...(Banino, Bartholomew) (Entered: 03/15/2023)
On April 11, the court orders PL to produce copies of the cases. https://www.courtlistener.com/docket/63107798/25/mata-v-avianca-inc/ Likewise, on April 12, the judge asks for another case. https://www.courtlistener.com/docket/63107798/27/mata-v-avianca-inc/
(This is not necessarily so unusual - some judges have local rules requiring litigants to attach unpublished cases to filings to save the court from having to go find them.)
ORDER: By April 18, 2022, Peter Lo Duca, counsel of record for plaintiff, shall file an affidavit annexing copies of the following cases cited in his submission to this Court: as set forth herein. Failure to comply will result in dismissal of the action pursuant to Rule 41 (b), Fed. R. Civ. P. SO ORDERED. (Signed by Judge P. Kevin Castel on 4/11/2023) (ama) (Entered: 04/11/2023)
PL asks for an extension. He's on vacation. https://www.courtlistener.com/docket/63107798/26/mata-v-avianca-inc/
Now...this seems a little odd to me. People should be able to take breaks, but really, you need more than a week to produce a copy of cases and an affidavit?
FIRST LETTER MOTION for Extension of Time to File Response/Reply as to 25 Order, addressed to Judge P. Kevin Castel from Peter LoDuca dated April 12, 2023. Document filed by Roberto Mata..(LoDuca, Peter) (Entered: 04/12/2023)
On April 25, PL files a response. It's amazing.
First, he says, he's attached the copies of the cases that he previously cited. (We'll come back to that.)
Second, that he couldn't find one of the cases that was cited by the court in one of those opinions.
Third, that the opinions are excerpts - that the attachments "may not be inclusive of the entire opinions but only what is made available by online database."
It's notarized by Stephen Schwartz.
https://www.courtlistener.com/docket/63107798/29/mata-v-avianca-inc/
RESPONSE in Opposition to Motion re: 16 MOTION to Dismiss pursuant to Fed. R. Civ. P. 12(b)(6). . Document filed by Roberto Mata. (Attachments: # 1 Exhibit Varghese v. China Southern Airlines, # 2 Exhibit Shaboon v. Egypt Air, # 3 Exhibit Peterson v. Iran Air, # 4 Exhibit Martinez v. Delta Airlines, # 5 Exhibit Estate of Durden v. KLM Royal Dutch Airlines, # 6 Exhibit Ehrlich v. American Airlines, # 7 Exhibit Miller v. United Airlines, Inc., # 8 Exhibit Ttivelloni-Lorenzi v. Pan American World Airways, Inc.).(LoDuca, Peter) (Entered: 04/25/2023)
PL then attaches the "opinions." Let's take one of them: https://storage.courtlistener.com/recap/gov.uscourts.nysd.575368/gov.uscourts.nysd.575368.29.1.pdf
First of all, this doesn't really look like a court opinion, formatting wise. There's no date.
And it's supposedly an 11th Circuit opinion. On page 2, it says "Before Jordan, Rosenbaum, and Higginbotham, *Circuit Judges."
One issue. Higginbotham isn't an 11th Circuit judge. There is a Patrick Higginbotham who is a federal judge. He served on the 5th Circuit, and took senior status before either of the two other judges (who are real) took office. https://en.wikipedia.org/wiki/Patrick_Higginbotham
Now, weird stuff happens in the federal courts, so it's not totally out of the question that someone might sit by designation or something else funky is happening. But still, that's very strange!
(Call back to when I went on @alex and @emilymbender's Mystery AI Hype Theater and talked about how the types of errors that generative AI makes are different in kind from the type that humans make.)
But also, this case is basically perfect. Like tailor made for the fact pattern of PL's case! Amazing that there's circuit level precedent that lines up with the lawyer's argument so cleanly.
The other ones are also pretty weird. There's parts that are in quotes (page 7)? There's a Texas state court referring to an 11th Circuit ruling as if they had made it (page 5)? https://storage.courtlistener.com/recap/gov.uscourts.nysd.575368/gov.uscourts.nysd.575368.29.4.pdf
So there's some stuff that should make a lawyer curious, in short.
[NB: I did not intend this to be this long but whatever, we're all nerds here.]
On May 4th, the shit really hits the fan. The judge issues an order to show cause why he shouldn't issue sanctions on PL for citing cases that don't exist and then swearing that the copies attached on April 25 are real. (Dkt #31)
This is one pissed off federal judge.
"Six of the submitted cases appear to be bogus judicial decisions with bogus quotes and bogus internal citations."
The judge called (or probably had his clerks call) the 11th Circuit and confirmed that the decision wasn't real. He also points out that the internal citations are also to "bogus" cases.
The full order is here: https://www.courtlistener.com/docket/63107798/31/mata-v-avianca-inc/
ORDER TO SHOW CAUSE: Let Peter LoDuca, counsel for plaintiff, show cause in person at 12 noon on June 8, 2023 in Courtroom 11D, 500 Pearl Street, New York, NY, why he ought not be sanctioned pursuant to: (1) Rule 11(b)(2) & (c), Fed. R. Civ. P., (2) 28 U.S.C. § 1927, and (3) the inherent power of the Court, for (A) citing non-existent cases to the Court in his Affirmation in Opposition (ECF 21), and (B) submitting to the Court annexed to his Affidavit filed April 25, 2023 copies of non-existent judicial opinions (ECF 29).2 Mr. LoDuca shall also file a written response to this Order by May 26, 2023. If Mr. LoDuca wishes to call live witnesses at the June 8, 2023 hearing, subject to cross-examination, he shall state so in his submission of May 26, identifying the name of the witness or witnesses. Defendant may respond by June 2, 2023. And as set forth herein. SO ORDERED. Show Cause Hearing set for 6/8/2023 at 12:00 PM in Courtroom 11D, 500 Pearl Street, New York, NY 10007 before Judge P. Kevin Castel. (Signed by Judge P. Kevin Castel on 5/04/2023) (ama) (Entered: 05/04/2023)
It turns out it was SS, not PL, who prepared the filings. PL just filed them, and in his own declaration, pleads ignorance.
Editor's note: My dude, you did have reason to doubt the authenticity of the case law. These were weirdass print outs that don't look like real cases and if you had literally plugged any of them into Westlaw or Lexis, you wouldn't have found them. https://storage.courtlistener.com/recap/gov.uscourts.nysd.575368/gov.uscourts.nysd.575368.32.0.pdf
The federal judge...is not mollified. (Nor should he be.) On May 26 (yesterday), he still orders the two lawyers to show cause why they, and their law firm, should not be sanctioned. https://www.courtlistener.com/docket/63107798/33/mata-v-avianca-inc/
I suspect that a significant part of the ire is about the doubling down on the cases in the April 25 filing. If they had come clean then, I feel like the judge might be more lenient.
ORDER TO SHOW CAUSE: This Order incorporates the Court's Order of May 4, 2023. Having received and reviewed the "Affidavit in Response to Order of May 4, 2023" filed by Peter LoDuca and the Affidavit of Steven A. Schwartz annexed thereto (ECF 32), the Court ORDERS as follows: Let Mr. LoDuca also show cause at the hearing of June 8, 2023 why he ought not be sanctioned pursuant to (1) Rule 11(b)(2) & (c), Fed. R. Civ. P., (2) 28 U.S.C. § 1927, and (3) the inherent power of the Court for the use of a false and fraudulent notarization in his affidavit filed on April 25, 2023 (ECF 29). Let the law firm of Levidow, Levidow & Oberman, P.C. (the "Levidow Firm"), show cause at the hearing of June 8, 2023 why it not be sanctioned pursuant to (1) Rule 11(b)(2) & (c), Fed. R. Civ. P., (2) 28 U.S.C. § 1927, and (3) the inherent power of the Court for the following: (A) the citation of non-existent cases to the Court in the Affirmation in Opposition filed on March 1, 2023 (ECF 21); (B) the submission to the Court of copies of non-existent judicial opinions annexed to the Affidavit filed on April 25, 2023; and (C) the use of a false and fraudulent notarization in the affidavit filed on April 25, 2023. Let Steven Schwartz show cause at the hearing of June 8, 2023 why he ought not be sanctioned pursuant to (1) Rule 11(b)(2) & (c), Fed. R. Civ. P., (2) 28 U.S.C. § 1927, and (3) the inherent power of the Court and referred to the Attorney Grievance Committee of the Appellate Division, First Department, and/or the Committee on Grievances of this District for aiding and causing (A) the citation of non-existent cases to the Court in the Affirmation in Opposition filed on March 1, 2023; (B) the submission to the Court of copies of non-existent judicial opinions annexed to the Affidavit filed on April 25, 2023; and (C) the use of a false and fraudulent notarization in the affidavit filed on April 25, 2023. Mr. LoDuca is ordered to serve a copy of this Order to Show Cause upon his employer, the law firm of Levidow, Levidow & Oberman, P.C., and upon Mr. Schwartz, who Mr. LoDuca states he intends to call as a witness at the hearing of June 8, 2023. Mr. Schwartz and the Levidow Firm shall submit a written response to this Order and Mr. LoDuca shall submit a supplemental written response to this Order by June 2, 2023. If Mr. LoDuca, Mr. Schwartz or the Levidow Firm wishes to call live witnesses at the June 8, 2023 hearing, subject to cross-examination, they shall identify them in writing by June 2, 2023. SO ORDERED. Show Cause Hearing set for 6/8/2023 at 12:00 PM in Courtroom 11D, 500 Pearl Street, New York, NY 10007 before Judge P. Kevin Castel. (Signed by Judge P. Kevin Castel on 5/26/2023) (ama) (Entered: 05/26/2023)
Oh, and remember how Steven Schwartz notarized the documents back at the end of April? Judge wants him to show cause as to why that wasn't fraudulent.
So first of all, PL basically lied in his affidavit, since SS was the one who produced the cases.
[Removed a comment here about the notary stamp, I stand corrected! Hooray for edits] https://social.coop/@adamgreenfield/110442030870549917.
@[email protected] Changing the date is, if not fine, at the very least a widespread practice. What matters is the date of witness and attestation thereof, and the commission number. 🤷♂️
Anyway, hearing is set for June 8th. Popcorn will be a must. Thanks for reading!
While you're here, check out the episode of Mystery AI Hype Theatre where I talk about why this is exactly the problem with generative AI hype in law. I think at about 30 minutes we talk about this problem. https://peertube.dair-institute.org/w/o6sb7f7RwapWBJd9VC61t4
@kendraserra here's a horrifying thought for you! I did work as a medical transcriptionist/editor for a while, and I can tell you that for a lot of doctors, a lot of their procedure/exam notes are virtually identical in a lot of ways. plug in the correct patient name and date of procedure/exam, and quite a few of them can be generated from templates. this is very much *not supposed to be done this way*, for many reasons, not least of which being that these notes go into a patient's PERMANENT MEDICAL RECORD and if they're inaccurate they can cause the patient (and other doctors, and insurance companies) a lot of serious problems down the road.
my question isn't whether some doctors have been sold ChatGPT to generate their own prodedure/exam notes quickly and cheaply; it's how many are *currently* doing this, how long they've been doing it, what it's going to take to get them caught and stop doing it, and how difficult will it be to find and weed out the ChatGPT records? >____<
@Lunaphied @troodon @kendraserra @VirginiaSOpossum There is some value regarding making medical notes inalterable (Except for inaccuracies admitted known of at the time of the notes being made.); in general, medical history notes may come up later at a time, and you want to know that you're not following up on something that was updated and didn't reference the note at that time.
There should be an edit history available though -
@troodon My worst experience with a doctor was when a psychiatrist asked me if I was going to kill myself. I said "no" but he wrote on my permanent medical record that I said "yes," and had me locked up against my will for being "dangerous to myself or others." He also wrote on my record that the reason I made the appointment was "suicidal ideation" when it was unrelated.
I wonder how often a LLM would make this mistake.
@kendraserra "Anyway, hearing is set for June 8th. I suspect popcorn will be a must." Indeed, I noticed that date - not too far off, and the place? Holy cow, that's like a 15-minute subway ride from me. Could I please ask your opinion: if I were to take a few hours off work and go to 500 Pearl St.,
1) Would I be able to watch?
2) Would it be interesting, or bore a non-lawyer silly?
Thanks!
.@kendraserra Just finished reading the thread and some of the docs. Wow. Wowwwwwwww.
Side note, as a former lawyer who swapped into the hard sciences, /this is why/ public accessibility of works cited is so essential.
Imagine if, instead of a court case where everybody involved is able to check these citations because they're all public records (leaving aside Westlaw and LexisNexis's iron grip over citation indexing), that this was a scientific paper someone "researched" using ChatGPT. If all the works cited are paywalled, it becomes a lot harder to check them. (Also, law citation standards require more precision than scientific citation standards, which forms another barrier to checking citations.)
@kendraserra this does raise questions about what will happen when not just one or two lawyers try this on, but swathes of those in the law profession decide to rely on generative "AI" to write cases.
The amount of time just trying to figure out if it's funkiness because mistakes happen, or funkiness because an "AI" has just make shit up, is going to clog the courts worse than they already are
@emilymbender many smart people are totally credulous when it comes to technology film flam. Things seriously smart people earnestly believe that are absolutely 180 degrees off from the truth: LLMs are AI (in any sense of the word), LLMs think, LLMs make decisions, LLMs can hallucinate, LLMs are new technology which will only improve.
Absolute bonkers bullshit. Wishful thinking, learned helplessness, enormous gullibility, with a huge helping of underlying christian eschatology.
@clauclauclaudia @edyoung @Seruko @emilymbender
Confabulating is the best word I've seen. Unconscious invention that you're unaware of inventing.
I like "Computationally expensive autocorr-wrong" because that's what it's doing. Taking a punt at what the answer looks like with no thought for detail.
If you haven't checked out #ThaiLibraryExperiment then that's a valuable exercise in empathy.
@edyoung @Seruko @emilymbender Engineers repurpose existing words all the time to get new ideas across to their peers and to non-experts. Sometimes those terms are evocative enough to make it into non-technical articles about the topic.
The problem isn't the terminology.
The problem is that there will always be humans who anthropomorphize -
a rock (Pet Rock)
a text editor (Eliza)
a robot (Sophia)
a machine learning model (ChatGPT).
@edyoung @Seruko @emilymbender
So what does a language model look like when it is "not hallucinating"?
Like, what is the output of a "lucid" language model?
@edyoung @Seruko @emilymbender Confabulation is a better term. In hallucination the viewer actually sees the non existent thing.
People who confabulate...range from subtle inaccuracies to surreal fabrications, and may include confusion or distortion in (timing, sequence or duration)...In general, they are very confident about their recollections, even when challenged with contradictory evidence.
https://en.m.wikipedia.org/wiki/Confabulation
You think that's bad, imagine what it would look like if the CDC used Palantir's temporally locked LLMs to confabulate the Biden Administration's pandemic response.
Actually, wait. That explains a lot.
@emilymbender @kendraserra It's bananas that anyone would think that using a statistical text predictor is a great idea for any research beyond research into LLMs and statistical text prediction.
Btw I think you are the one with the Thai analogy. You might want to add that as often, computers are dramatically better at tasks that they can do, so slurping up the Thai national library and pretending to read/write Thai, would probably be a snap for a LLM.