A man sued Avianca Airline – his lawyer used ChatGPT

The lawyer kept digging the hole deeper and deeper, and (as a non-expert) I agree that it seems that the lawyer is at serious risk of being disbarred.

Interesting documents are from #24 onwards:

- #24 (https://storage.courtlistener.com/recap/gov.uscourts.nysd.57...): "unable to locate most of the case law cited in Plaintiff’s Affirmation in Opposition, and the few cases which the undersigned has been able to locate do not stand for the propositions for which they are cited"

- #25 (https://storage.courtlistener.com/recap/gov.uscourts.nysd.57...) & #27 (https://storage.courtlistener.com/recap/gov.uscourts.nysd.57...): order to affix copies of cited cases

- #29: attached the cases - later revealed to be a mixture of made up (bogus) for some, vs irrelevant for others

- #30 (https://storage.courtlistener.com/recap/gov.uscourts.nysd.57...): "the authenticity of many of these cases is questionable" - polite legal speak for bogus. And "these cases do exist but submits that they address issues entirely unrelated to the principles for which Plaintiff cited them" - irrelevant. And a cutting aside that "(The Ehrlich and In re Air Crash Disaster cases are the only ones submitted in a conventional format.)" - drawing attention to the smoking gun for the bogus cases

- #31 (https://storage.courtlistener.com/recap/gov.uscourts.nysd.57...): an unhappy federal judge: "The Court is presented with an unprecedented circumstance. A submission filed by plaintiff’s counsel in opposition to a motion to dismiss is replete with citations to non-existent cases. ... Six of the submitted cases appear to be bogus judicial decisions with bogus quotes and bogus internal citations" ---- this PDF is worth reading in full, it is only 3 pages & excoriating

- #32 affidavits, including the ChatGPT screenshot

- #33 (https://storage.courtlistener.com/recap/gov.uscourts.nysd.57...): an even more unhappy judge: invitation for the lawyer & law firm to explain why they "ought not be sanctioned"

https://www.thestar.com/news/gta/2023/05/26/tdsb-fires-resea...

Let’s not forget that Google often puts incorrect information in their snippets/factboxes or whatever they call them.

> This scientific description provides an overview of the formula and manufacturing process of transparent aluminum, a material used in applications where both structural strength and transparency are required. Transparent aluminum finds extensive use in diverse fields, including public aquaria, where it allows for the display of large marine organisms. The description outlines the chemical composition, key properties, and the manufacturing steps involved in creating transparent aluminum.

Whether or not the six-step manufacturing process it came up with is correct or not, I haven't the expertise to say.

I imagine at some point this behaviour will be built in. People are treating GPTs like they're knowledge bases, but they're not. The fact that they can answer simple or complex questions correctly is only a byproduct of the transformer being taught how to string language together.

It's like an artist who learns to draw by making paintings of raccoons. You can't then ask them "what do raccoons like to eat?" or "what foods are poisonous to a raccoon?" just because they learnt to draw by painting raccoons. This is how people are treating GPTs atm. They believe in it because they ask the artist "what colour fur do raccoons have?" and because the artist can answer that correctly, they assume all other answers are factual.

So you're trying to say that lawyering is the hardest job in the world and only the smartest people can do it...?

At least one company already integrates or is about to (not sure if it's in production yet) LLMs in theirs to do effectively smarter completions.

It won't be fully automated any time soon, but it will certainly eat into a lot of the simpler work.

Certainly, civil and criminal courts have those as their raison d'être. But I thought licensing boards had an entirely different purpose. If I surgeon was a good guy who genuinely wanted to help people and who didn't engage in any sort of malfeasance... but even so, he just kept slicing aortas open accidentally through incompetence, the board should say "aw shucks, he's had some bad luck but he really wants to heal people".

This is the same. The court system is replete with circumstances where a client does not get a second chance at pursuing justice. A lawyer that fucks that up, even if doing so in good faith, leaves them with zero remedies. This might have been a bullshit "Slippin' Jimmy" case this time, but the stakes could've easily been higher.

I don't think I want to live in a world where fairness plays any part in the decision by the bar on this matter.

I don't think this is indictiative of a wider issue. Or likely to be repeated substantially.

In terms of severity it's painfully foolish but then again ChatGPT is a totally new tool and a lot of people will be caught off guard.

I am stunned the lawyer didn't at least look up the case notes or even prepare a pocket brief if he believed they were real but hard to find

Why did he think it was true, if he had never used it before then?

"I found it online" levels of competence here.

When the lawyer brought evidence that he had tried to verify the information (B), that evidence itself automatically disproves his plead that he didn't think it could be false (A).

(Note that didn't have to necessarily be the case: for example, if he had claimed that it was his assistant who asked the verification question, rather than he himself.)

So, unless he made the second plead and brought the evidence at a latter time, shouldn't he have just skipped A altogether, and preserved his credibility? (And possibly avoided a perjury charge? IDK if in an American court both claims would be sworn statements)

I know it is a bad bias, but we typically associate good and clear writing with legitimacy. Here we have chat GPT, that can do exactly that, but spit out complete bs.

Why he didn't verify the existence of these cases spit out by a tool he used for the first time is beyond me. It would have taken him a few minutes at most to discover the fake ones and skimming the bogus references (assuming some legal competence)

It's crazy how AI seems to progress at incredible speed and yet we don't get closer to full autonomy anywhere. It's as if we discover new problems at the same speed we are solving them. Just 5 years ago nobody would think hallucinations will become a central issue in AI, we might discover other unknown unknowns that hide in our future.

However it's always interesting to watch how people react: * Traditional programming creates very strict functionality but is not flexible/is rigid. * Okay, so let's train models to act like biological life, which is super flexible but has a chance to make mistakes.

But then people complain about the latter making mistakes. Realistically we can combine the two to ensure that what a model is saying is based on fact, though.

Easiest way would be just for people to interact with an LLM to browse hard facts and then double check via sources. But relying on humans not to be lazy is just as good as relying on an ML model to act like traditional programming, so it's probably better to train our wild model with as many built-in constraints as possible and then decorate those with more traditionally programmed hard limits.

It's really hard though because of the output data. We're going from a binary output of 1/0 for traditional programming to a continuous output of 0-1 for models; if I ask it to summarise something, whether it's correct or not is not only based on whether it summarised based only on the original text, but also down to the reviewer's individual biases and wants/needs as to whether the summarisation is sufficient.

The continuous nature of an LLM's response does make it difficult to determine if it's sufficiently factually correct, though. Because if it's not repeating word for word, you need to be able to parse the output in order to check it...using another LLM (so compounding error...)

UNITED STATES SUPREME COURT

Saul Goodman's Legal Advocacy Clinic 123 Shady Lane Albuquerque, NM 87102 Phone: (555) 123-4567 Email: saulgoodman@lawyerextraordinaire.com

May 27, 2023

JUSTICES OF THE SUPREME COURT

Re: Amicus Curiae Brief - Case No. 12345

Dear Honorable Justices,

I, Saul Goodman, attorney-at-law and mastermind of legal brilliance, on behalf of the humble but feisty David Rosencrantz, submit this amicus curiae brief to the esteemed Supreme Court. This Court, known for its unflinching pursuit of justice, now has the opportunity to rectify a grave miscarriage of justice perpetrated by the lower courts.

SUMMARY OF ARGUMENT

The case before this honorable Court presents a classic tale of a downtrodden individual, David Rosencrantz, pitted against the formidable Goliath that is the United States government. Rosencrantz, a simple taxi driver trying to earn an honest buck in this dog-eat-dog world, has been unjustly accused of tax evasion. However, this Amicus respectfully submits that the government's case is nothing but smoke and mirrors, an elaborate scheme to crush the dreams of an honest, hardworking citizen.

ARGUMENT

I. Violation of Due Process

The lower courts, in their zealous pursuit of victory, trampled upon the sacred principles of due process. Mr. Rosencrantz was denied his fundamental right to a fair trial when the government's star witness, a notorious jailbird with a penchant for singing, was allowed to testify without any corroborating evidence. The government's reliance on the word of a jailhouse informant, who likely had ulterior motives, is an affront to the Constitution.

II. Reasonable Doubt

Ladies and gentlemen of the Court, this case is riddled with reasonable doubt like a cheap suit in a thrift store. The government's case rests on flimsy evidence and a convoluted web of speculation. The alleged tax discrepancies are based on questionable calculations and fuzzy math, all while conveniently ignoring legitimate deductions and expenses rightfully claimed by Mr. Rosencrantz. Reasonable doubt casts its long shadow on this entire proceeding, and it is the duty of this Court to bring it into the light.

III. Selective Prosecution

This Court has never turned a blind eye to injustice, and I implore you to scrutinize the government's motives. Mr. Rosencrantz, an unassuming member of society, has become the target of selective prosecution. Why, might you ask? It's because he dared to stand up against the establishment, questioning the powers that be. This case is not about taxes; it is about silencing dissent and punishing those who refuse to play by their rigged rules.

CONCLUSION

In conclusion, the government's case against David Rosencrantz reeks of a witch hunt, an attempt to crush the spirit of an ordinary citizen who dares to question authority. The violation of due process, the presence of reasonable doubt, and the specter of selective prosecution are grave injustices that cannot be ignored. This Court, armed with the sword of justice, must rise above the fog of deceit and deliver a resounding verdict in favor of Mr. Rosencrantz.

For these reasons, I, Saul Goodman, implore this Court to right the wrongs committed by the lower courts and restore faith in the sanctity of our legal system.

Respectfully submitted,

Saul Goodman, Esq. Attorney for Amicus Curiae Saul Goodman's Legal Advocacy Clinic