Are you just catching up on the bonkers story about the lawyer using ChatGPT for federal court filings? This is a thread for you.

Our dramatis personae - some lawyers in federal court, in a lawsuit over a personal injury on an airplane.
Bartholomew Banino (BB) represents the airline.

Peter LoDuca (PL) and Steven Schwartz (SS) represent the injured person.

Here's the docket (via @questauthority) https://www.courtlistener.com/docket/63107798/mata-v-avianca-inc/?filed_after=&filed_before=&entry_gte=&entry_lte=&order_by=asc

Mata v. Avianca, Inc., 1:22-cv-01461 - CourtListener.com

Docket for Mata v. Avianca, Inc., 1:22-cv-01461 — Brought to you by Free Law Project, a non-profit dedicated to creating high quality open legal information.

CourtListener
The case is in federal court, and they're arguing over whether it should be there or in state court. The airline has filed its motion to dismiss on Jan. 13. On Jan. 18, the plaintiff asks for more time to reply (#19). The judge gives it to them. (#20).

On March 1st, PL files that reply/opposition to the motion to dismiss.

It cites a bunch of cases in support of its argument! They seem quite convincing - there's some state courts holding that state courts can decide international airline accidents. (See pages 4 and 5). https://storage.courtlistener.com/recap/gov.uscourts.nysd.575368/gov.uscourts.nysd.575368.21.0.pdf

Transcript Order – #663 in In re: Google Play Store Antitrust Litigation (N.D. Cal., 3:21-md-02981) – CourtListener.com

Transcript Order

CourtListener
The defendants file their response. In it, there's a footnote where they flag that they can't find the cases that PL cited to, or they don't say what PL said they said. https://www.courtlistener.com/docket/63107798/24/mata-v-avianca-inc/
Reply Memorandum of Law in Support of Motion – #24 in Mata v. Avianca, Inc. (S.D.N.Y., 1:22-cv-01461) – CourtListener.com

REPLY MEMORANDUM OF LAW in Support re: 16 MOTION to Dismiss pursuant to Fed. R. Civ. P. 12(b)(6). . Document filed by Avianca, Inc...(Banino, Bartholomew) (Entered: 03/15/2023)

CourtListener

On April 11, the court orders PL to produce copies of the cases. https://www.courtlistener.com/docket/63107798/25/mata-v-avianca-inc/ Likewise, on April 12, the judge asks for another case. https://www.courtlistener.com/docket/63107798/27/mata-v-avianca-inc/

(This is not necessarily so unusual - some judges have local rules requiring litigants to attach unpublished cases to filings to save the court from having to go find them.)

Order – #25 in Mata v. Avianca, Inc. (S.D.N.Y., 1:22-cv-01461) – CourtListener.com

ORDER: By April 18, 2022, Peter Lo Duca, counsel of record for plaintiff, shall file an affidavit annexing copies of the following cases cited in his submission to this Court: as set forth herein. Failure to comply will result in dismissal of the action pursuant to Rule 41 (b), Fed. R. Civ. P. SO ORDERED. (Signed by Judge P. Kevin Castel on 4/11/2023) (ama) (Entered: 04/11/2023)

CourtListener

PL asks for an extension. He's on vacation. https://www.courtlistener.com/docket/63107798/26/mata-v-avianca-inc/

Now...this seems a little odd to me. People should be able to take breaks, but really, you need more than a week to produce a copy of cases and an affidavit?

Motion for Extension of Time to File Response/Reply – #26 in Mata v. Avianca, Inc. (S.D.N.Y., 1:22-cv-01461) – CourtListener.com

FIRST LETTER MOTION for Extension of Time to File Response/Reply as to 25 Order, addressed to Judge P. Kevin Castel from Peter LoDuca dated April 12, 2023. Document filed by Roberto Mata..(LoDuca, Peter) (Entered: 04/12/2023)

CourtListener
The judge gives PL more time (and apparently rope to hang himself). We're now up to April and docket number 28.

On April 25, PL files a response. It's amazing.

First, he says, he's attached the copies of the cases that he previously cited. (We'll come back to that.)

Second, that he couldn't find one of the cases that was cited by the court in one of those opinions.

Third, that the opinions are excerpts - that the attachments "may not be inclusive of the entire opinions but only what is made available by online database."

It's notarized by Stephen Schwartz.

https://www.courtlistener.com/docket/63107798/29/mata-v-avianca-inc/

Response in Opposition to Motion – #29 in Mata v. Avianca, Inc. (S.D.N.Y., 1:22-cv-01461) – CourtListener.com

RESPONSE in Opposition to Motion re: 16 MOTION to Dismiss pursuant to Fed. R. Civ. P. 12(b)(6). . Document filed by Roberto Mata. (Attachments: # 1 Exhibit Varghese v. China Southern Airlines, # 2 Exhibit Shaboon v. Egypt Air, # 3 Exhibit Peterson v. Iran Air, # 4 Exhibit Martinez v. Delta Airlines, # 5 Exhibit Estate of Durden v. KLM Royal Dutch Airlines, # 6 Exhibit Ehrlich v. American Airlines, # 7 Exhibit Miller v. United Airlines, Inc., # 8 Exhibit Ttivelloni-Lorenzi v. Pan American World Airways, Inc.).(LoDuca, Peter) (Entered: 04/25/2023)

CourtListener

PL then attaches the "opinions." Let's take one of them: https://storage.courtlistener.com/recap/gov.uscourts.nysd.575368/gov.uscourts.nysd.575368.29.1.pdf

First of all, this doesn't really look like a court opinion, formatting wise. There's no date.

Transcript Order – #663 in In re: Google Play Store Antitrust Litigation (N.D. Cal., 3:21-md-02981) – CourtListener.com

Transcript Order

CourtListener

And it's supposedly an 11th Circuit opinion. On page 2, it says "Before Jordan, Rosenbaum, and Higginbotham, *Circuit Judges."

One issue. Higginbotham isn't an 11th Circuit judge. There is a Patrick Higginbotham who is a federal judge. He served on the 5th Circuit, and took senior status before either of the two other judges (who are real) took office. https://en.wikipedia.org/wiki/Patrick_Higginbotham

Patrick Higginbotham - Wikipedia

Now, weird stuff happens in the federal courts, so it's not totally out of the question that someone might sit by designation or something else funky is happening. But still, that's very strange!

(Call back to when I went on @alex and @emilymbender's Mystery AI Hype Theater and talked about how the types of errors that generative AI makes are different in kind from the type that humans make.)

But also, this case is basically perfect. Like tailor made for the fact pattern of PL's case! Amazing that there's circuit level precedent that lines up with the lawyer's argument so cleanly.

The other ones are also pretty weird. There's parts that are in quotes (page 7)? There's a Texas state court referring to an 11th Circuit ruling as if they had made it (page 5)? https://storage.courtlistener.com/recap/gov.uscourts.nysd.575368/gov.uscourts.nysd.575368.29.4.pdf

So there's some stuff that should make a lawyer curious, in short.

Transcript Order – #663 in In re: Google Play Store Antitrust Litigation (N.D. Cal., 3:21-md-02981) – CourtListener.com

Transcript Order

CourtListener
On April 26, the day after the affidavit with he cases is filed, BB, the lawyer for the defendant, files a letter with questions. They literally can't find the cases anywhere else. The docket numbers don't line up. And the federal reporters (sort of like a DOI, for my non-lawyer readers) turn up a different case. https://storage.courtlistener.com/recap/gov.uscourts.nysd.575368/gov.uscourts.nysd.575368.30.0.pdf
Transcript Order – #663 in In re: Google Play Store Antitrust Litigation (N.D. Cal., 3:21-md-02981) – CourtListener.com

Transcript Order

CourtListener

[NB: I did not intend this to be this long but whatever, we're all nerds here.]

On May 4th, the shit really hits the fan. The judge issues an order to show cause why he shouldn't issue sanctions on PL for citing cases that don't exist and then swearing that the copies attached on April 25 are real. (Dkt #31)

This is one pissed off federal judge.

"Six of the submitted cases appear to be bogus judicial decisions with bogus quotes and bogus internal citations."

The judge called (or probably had his clerks call) the 11th Circuit and confirmed that the decision wasn't real. He also points out that the internal citations are also to "bogus" cases.

The full order is here: https://www.courtlistener.com/docket/63107798/31/mata-v-avianca-inc/

Order to Show Cause – #31 in Mata v. Avianca, Inc. (S.D.N.Y., 1:22-cv-01461) – CourtListener.com

ORDER TO SHOW CAUSE: Let Peter LoDuca, counsel for plaintiff, show cause in person at 12 noon on June 8, 2023 in Courtroom 11D, 500 Pearl Street, New York, NY, why he ought not be sanctioned pursuant to: (1) Rule 11(b)(2) & (c), Fed. R. Civ. P., (2) 28 U.S.C. § 1927, and (3) the inherent power of the Court, for (A) citing non-existent cases to the Court in his Affirmation in Opposition (ECF 21), and (B) submitting to the Court annexed to his Affidavit filed April 25, 2023 copies of non-existent judicial opinions (ECF 29).2 Mr. LoDuca shall also file a written response to this Order by May 26, 2023. If Mr. LoDuca wishes to call live witnesses at the June 8, 2023 hearing, subject to cross-examination, he shall state so in his submission of May 26, identifying the name of the witness or witnesses. Defendant may respond by June 2, 2023. And as set forth herein. SO ORDERED. Show Cause Hearing set for 6/8/2023 at 12:00 PM in Courtroom 11D, 500 Pearl Street, New York, NY 10007 before Judge P. Kevin Castel. (Signed by Judge P. Kevin Castel on 5/04/2023) (ama) (Entered: 05/04/2023)

CourtListener
On May 4th, SS admits what happened (which you probably already figured out, even if you missed the header toot). It was ChatGPT. The declaration here is just too amazing not to provide you with it directly. https://storage.courtlistener.com/recap/gov.uscourts.nysd.575368/gov.uscourts.nysd.575368.32.1.pdf
Transcript Order – #663 in In re: Google Play Store Antitrust Litigation (N.D. Cal., 3:21-md-02981) – CourtListener.com

Transcript Order

CourtListener

It turns out it was SS, not PL, who prepared the filings. PL just filed them, and in his own declaration, pleads ignorance.

Editor's note: My dude, you did have reason to doubt the authenticity of the case law. These were weirdass print outs that don't look like real cases and if you had literally plugged any of them into Westlaw or Lexis, you wouldn't have found them. https://storage.courtlistener.com/recap/gov.uscourts.nysd.575368/gov.uscourts.nysd.575368.32.0.pdf

Transcript Order – #663 in In re: Google Play Store Antitrust Litigation (N.D. Cal., 3:21-md-02981) – CourtListener.com

Transcript Order

CourtListener

The federal judge...is not mollified. (Nor should he be.) On May 26 (yesterday), he still orders the two lawyers to show cause why they, and their law firm, should not be sanctioned. https://www.courtlistener.com/docket/63107798/33/mata-v-avianca-inc/

I suspect that a significant part of the ire is about the doubling down on the cases in the April 25 filing. If they had come clean then, I feel like the judge might be more lenient.

Order to Show Cause – #33 in Mata v. Avianca, Inc. (S.D.N.Y., 1:22-cv-01461) – CourtListener.com

ORDER TO SHOW CAUSE: This Order incorporates the Court's Order of May 4, 2023. Having received and reviewed the "Affidavit in Response to Order of May 4, 2023" filed by Peter LoDuca and the Affidavit of Steven A. Schwartz annexed thereto (ECF 32), the Court ORDERS as follows: Let Mr. LoDuca also show cause at the hearing of June 8, 2023 why he ought not be sanctioned pursuant to (1) Rule 11(b)(2) & (c), Fed. R. Civ. P., (2) 28 U.S.C. § 1927, and (3) the inherent power of the Court for the use of a false and fraudulent notarization in his affidavit filed on April 25, 2023 (ECF 29). Let the law firm of Levidow, Levidow & Oberman, P.C. (the "Levidow Firm"), show cause at the hearing of June 8, 2023 why it not be sanctioned pursuant to (1) Rule 11(b)(2) & (c), Fed. R. Civ. P., (2) 28 U.S.C. § 1927, and (3) the inherent power of the Court for the following: (A) the citation of non-existent cases to the Court in the Affirmation in Opposition filed on March 1, 2023 (ECF 21); (B) the submission to the Court of copies of non-existent judicial opinions annexed to the Affidavit filed on April 25, 2023; and (C) the use of a false and fraudulent notarization in the affidavit filed on April 25, 2023. Let Steven Schwartz show cause at the hearing of June 8, 2023 why he ought not be sanctioned pursuant to (1) Rule 11(b)(2) & (c), Fed. R. Civ. P., (2) 28 U.S.C. § 1927, and (3) the inherent power of the Court and referred to the Attorney Grievance Committee of the Appellate Division, First Department, and/or the Committee on Grievances of this District for aiding and causing (A) the citation of non-existent cases to the Court in the Affirmation in Opposition filed on March 1, 2023; (B) the submission to the Court of copies of non-existent judicial opinions annexed to the Affidavit filed on April 25, 2023; and (C) the use of a false and fraudulent notarization in the affidavit filed on April 25, 2023. Mr. LoDuca is ordered to serve a copy of this Order to Show Cause upon his employer, the law firm of Levidow, Levidow & Oberman, P.C., and upon Mr. Schwartz, who Mr. LoDuca states he intends to call as a witness at the hearing of June 8, 2023. Mr. Schwartz and the Levidow Firm shall submit a written response to this Order and Mr. LoDuca shall submit a supplemental written response to this Order by June 2, 2023. If Mr. LoDuca, Mr. Schwartz or the Levidow Firm wishes to call live witnesses at the June 8, 2023 hearing, subject to cross-examination, they shall identify them in writing by June 2, 2023. SO ORDERED. Show Cause Hearing set for 6/8/2023 at 12:00 PM in Courtroom 11D, 500 Pearl Street, New York, NY 10007 before Judge P. Kevin Castel. (Signed by Judge P. Kevin Castel on 5/26/2023) (ama) (Entered: 05/26/2023)

CourtListener

Oh, and remember how Steven Schwartz notarized the documents back at the end of April? Judge wants him to show cause as to why that wasn't fraudulent.

So first of all, PL basically lied in his affidavit, since SS was the one who produced the cases.

[Removed a comment here about the notary stamp, I stand corrected! Hooray for edits] https://social.coop/@adamgreenfield/110442030870549917.

Adam Greenfield (@[email protected])

@[email protected] Changing the date is, if not fine, at the very least a widespread practice. What matters is the date of witness and attestation thereof, and the commission number. 🤷‍♂️

social.coop

Anyway, hearing is set for June 8th. Popcorn will be a must. Thanks for reading!

While you're here, check out the episode of Mystery AI Hype Theatre where I talk about why this is exactly the problem with generative AI hype in law. I think at about 30 minutes we talk about this problem. https://peertube.dair-institute.org/w/o6sb7f7RwapWBJd9VC61t4

Mystery AI Hype Theater, Episode 10 - Don't Be A Lawyer, ChatGPT

PeerTube

Update on the ChatGPT lawyer saga: in a fit of wisdom, Steven Schwartz, the lawyer who plugged things into ChatGPT, and his firm have lawyered up.

An expert on attorney negligence, you say? Seems like a good choice.

@kendraserra This was quite an adventure of a read! Thanks for posting!

@kendraserra here's a horrifying thought for you! I did work as a medical transcriptionist/editor for a while, and I can tell you that for a lot of doctors, a lot of their procedure/exam notes are virtually identical in a lot of ways. plug in the correct patient name and date of procedure/exam, and quite a few of them can be generated from templates. this is very much *not supposed to be done this way*, for many reasons, not least of which being that these notes go into a patient's PERMANENT MEDICAL RECORD and if they're inaccurate they can cause the patient (and other doctors, and insurance companies) a lot of serious problems down the road.

my question isn't whether some doctors have been sold ChatGPT to generate their own prodedure/exam notes quickly and cheaply; it's how many are *currently* doing this, how long they've been doing it, what it's going to take to get them caught and stop doing it, and how difficult will it be to find and weed out the ChatGPT records? >____<

@kendraserra Thanks for the great thread Kendra!
@kendraserra I love point 10 in the defense. “had no idea it makes stuff up”.
@kendraserra This all thing is surreal. And, in the end, I feel bad for the plaintiff whose lawyer derailed their case by making up citations.

@kendraserra "Anyway, hearing is set for June 8th. I suspect popcorn will be a must." Indeed, I noticed that date - not too far off, and the place? Holy cow, that's like a 15-minute subway ride from me. Could I please ask your opinion: if I were to take a few hours off work and go to 500 Pearl St.,

1) Would I be able to watch?
2) Would it be interesting, or bore a non-lawyer silly?

Thanks!

.@kendraserra Just finished reading the thread and some of the docs. Wow. Wowwwwwwww.

Side note, as a former lawyer who swapped into the hard sciences, /this is why/ public accessibility of works cited is so essential.

Imagine if, instead of a court case where everybody involved is able to check these citations because they're all public records (leaving aside Westlaw and LexisNexis's iron grip over citation indexing), that this was a scientific paper someone "researched" using ChatGPT. If all the works cited are paywalled, it becomes a lot harder to check them. (Also, law citation standards require more precision than scientific citation standards, which forms another barrier to checking citations.)

@kendraserra thank you so much for the thread, I've been curious about this clownshow but didn't find time to research! Hope you don't mind a follow :)
@kendraserra I'm not a lawyer so I am wondering how much information you would have to provide to chat GPT to come up with a similar brief.

@kendraserra this does raise questions about what will happen when not just one or two lawyers try this on, but swathes of those in the law profession decide to rely on generative "AI" to write cases.

The amount of time just trying to figure out if it's funkiness because mistakes happen, or funkiness because an "AI" has just make shit up, is going to clog the courts worse than they already are

@kendraserra Thank you for writing this up! I enjoyed the read.
@kendraserra oh wow, this is actively defrauding the court and they'd be looking at suspensions at least I believe--the original error was unforgivably sloppy lawyering but at least it was ChatGPT who did the lying (which is what it does! Its output should have been double-checked!), but like you said, the attempted cover-up is even worse.
@kendraserra Oh my, thank you for putting this thread together! This was interesting and rather entertaining. Can't wait for the 'find out' part now!

@kendraserra This reminds of one lawyer, the entire case about him was made up about him.

It makes me wonder how deep that rabbit hole goes with bad case filing.

@kendraserra Made this a couple days ago with some of the words from the brief https://objection.lol/objection/5188022 #AceAttorney #PhoenixWright
Ace Attorney Objection Maker

An online ace attorney case maker and generator.

@kendraserra LMAO..... I KNEW something like this would happen. The only thing I wish this had included was one of those fake cases citing an Ace Attorney case - but AHAHAHAHAHAHAHHAHAHAHA.

Hopefully this keeps ChatGPT out of court for a while.

(Also, those lawyers? The person they're representing can now sue them for legal malpractice and more...)

@kendraserra

I think what the Judge very specifically asked is:

How did you notarize with a date of January 25th 2023 a reply submitted in April 2023 that I didn't even ask for until April 11, 2023?

That's the kind of mistake notaries are never supposed to make and their procedures should prevent, because dates of legal documents *are* important (and a lawyer of all people should know that.) Hence the demand to bring his logbook and original of the signed document.

Thank you for this amazing thread @kendraserra ! It's bananas that anyone would think using ChatGPT for legal research was a good idea. But reading your expert analysis of the situation is a treat :)

@emilymbender many smart people are totally credulous when it comes to technology film flam. Things seriously smart people earnestly believe that are absolutely 180 degrees off from the truth: LLMs are AI (in any sense of the word), LLMs think, LLMs make decisions, LLMs can hallucinate, LLMs are new technology which will only improve.

Absolute bonkers bullshit. Wishful thinking, learned helplessness, enormous gullibility, with a huge helping of underlying christian eschatology.

@Seruko @emilymbender I think "hallucinating" is a useful term of art for when a language model invents references to non-existent things (like legal cases) which ostensibly exist in the real world. Obviously not the same thing as humans hallucinating.

@Seruko @emilymbender

You think that's bad, imagine what it would look like if the CDC used Palantir's temporally locked LLMs to confabulate the Biden Administration's pandemic response.

Actually, wait. That explains a lot.

Latest version of ChatGPT aces bar exam with score nearing 90th percentile

ABA Journal

@emilymbender @kendraserra It's bananas that anyone would think that using a statistical text predictor is a great idea for any research beyond research into LLMs and statistical text prediction.

Btw I think you are the one with the Thai analogy. You might want to add that as often, computers are dramatically better at tasks that they can do, so slurping up the Thai national library and pretending to read/write Thai, would probably be a snap for a LLM.

@kendraserra Outstanding summary. Thank you.

(Also calling @nikolausf )

@kendraserra What a ride. Thank you for the entertaining read!

My take away: if there's a shortcut, people will take it for better or worse. What a mess!

@kendraserra

Wow. I legitimately can't think of a stupider way to get in trouble in court. XD

@kendraserra Thank you! The whole thing is *wild* given how learned the representation is supposed to be, how polluting AI seems to be for EVERYTHING as well as how gullible we are when ‘sources’ are quoted 🤦🏾‍♀️
@kendraserra I am already writing the notes and questions on this one for my casebook in my head.
@jtlg It is literally basically what skeptics predicted would happen, playing out on a federal docket.
@kendraserra My dude, /my dude/, do you not know what being the attorney of record means, you are putting your own name on the line and saying you /at least/ conducted minimal review of what you're submitting, good catgods. (In general, anyway, idk NY law specifically.)
@kendraserra "I didn't actually do any research, but I did ask ChatGPT, which gave me some made up citations and opinions, and I just copy-pasted them without even checking that they were real. I'm very very sorry, and I won't do it again."
@kendraserra People really need to realize that a *text generator* is not a *search engine* .
@Sylvhem @kendraserra
Agreed. And companies (looking at you, Microsoft and Bing) need to stop thinking that a text generator can replace a search engine.
@kendraserra I remember people being fooled by things that seem obvious when the Internet was new. To people new to technology, it seemed like it took a lot of effort publish something.
The same thing is happening with these language models. The technology is amazing, esp if it keeps improving. It will take society a while to learn to use new tools based on it well. I think it's exciting.
@cgervasi @kendraserra it's "exciting" until it's *your* lawyer or doctor using ChatGPT. if you're lucky it gets caught and stopped, like with this lawyer. if you're not lucky, it ends up in your permanent medical (or legal) record and causes problems years later when you need critical treatment, as is probably happening in a lot of places right now. then it's a whole different definition of "exciting."
@troodon Risk of misuse is there for any exciting new technology. @kendraserra
@cgervasi @troodon @kendraserra helped along by a massive helping of unwarranted, breathless hype about the new technology, misleading people as to the possibilities and risks. The bullshit merchants bear as much moral culpability for this lawyer trying to play fast and loose as the lawyer themselves.