Two US lawyers fined for submitting fake court citations from ChatGPT
theguardian.com
theguardian.com
They lied to a judge, doubled down, showed further incompetence by admitting to not knowing how the legal reference they supposedly quoted was even structured, then lied to a judge again trying to get more time to get their story straight, and all it cost them was $5000? And even then they have the guts to say they disagree with the judge?
Levidow, Levidow & Oberman must either be run by fools or take the rest of the world for fools.
I can understand how the rubber stamping lawyer signing all the paperwork got off easy; he was just signing documents because the lawyer who supposedly did all the work for the case couldn't operate in the court the case was transfered to.
The other lawyer, the one who did the AI equivalent from copy pasting from the summary page of Google, should not be deemed fit to practice law.
They also have to write written apologies to their client, and all of the judges who were mentioned in their fake cases, with proof filed in the docket.
And they are the brunt of a very well-publicized lawsuit. The judge is effectively saying that all that publicity is punishment enough; he doesn't need to do anymore. He doesn't need to refer them to the bar for disbarment--the bar already knows about it. (The judge has no power to disbar them himself, he can only refer to the bar for disbarment.)
The biggest surprise is that they didn't get made to pay the defendant's legal fees as well, but that's because defendant didn't ask for any fees.
Oh, and the final punishment... they lost the case--they get no chance to redo their opposition to the motion to dismiss. Arguably, their client can also sue them for malpractice, and their malpractice carrier is likely to prefer to settle that rather than actually fight the case.
This isn't going to go away. This decision will dog them for the entire rest of their careers.
I get the impression from the events leading up to the use of GPT this is not the first time their competence led to outcomes that will "dog them for the entire rest of their careers."
I thought the judge said he couldn't force them to do that because it wouldn't be sincere.
They'll probably just have ChatGPT write their apologies for them.
As an AI language model...
I have been a good Bing.
"The April 25 affidavit was filed in response to the Orders of April 11 and 12, 2023, but is sworn to before a Notary Public on the 25th of January 2023. Mr. LoDuca should be prepared to address this anomaly on June 8, 2023 and bring with him a wet-ink signed copy of the affidavit with the notary’s signature and stamp."
a reference from the judge and the bar just knowing about it are very different.
Well, no, they also have to right letters and attach the sanctions order, and the transcript of the sanctions hearing to:
(1) their client in the case, and
(2) each of the real judges to whom they (on the basis of ChatGPT “research”) falsely attributed a decision.
But, yes, sanctions are fairly light (particularly the absence of a disciplinary referral).
> I can understand how the rubber stamping lawyer signing all the paperwork got off easy; he was just signing documents because the lawyer who supposedly did all the work for the case couldn't operate in the court the case was transfered to.
> The other lawyer, the one who did the AI equivalent from copy pasting from the summary page of Google, should not be deemed fit to practice law.
They both failed their duty in nearly identical ways; passing of someone else’s work to which they did not apply independent professional analysis as their professional work. If anything, the one doing so directly to the court should be more, not less, liable to outside sanction (if the issue was sanctions within the firm, that’s a different story, maybe.)
Once they showed up in court, one of them was unable to explain how to interpret a very basic "you should know this to practice law" case citation ("F.3d"[1]). Not only did they not "apply independent professional analysis" it seems they were woefully unprepared to even produce such a thing.
Sorry to be cynical, but this sounds like they delegated their defense to AI as well... in the near future this kind of argument will probably work in a lot of contexts.
God help us when it helps in court.
Why should you get off easy?
Seems like they still don't get it.
It's not good faith representation of your client, it's not good lawyering to interrogate a chatbot for your case. Even if it could somehow only ever tell the truth.
It's not a "good faith mistake" - so I think they still don't understand the problem here.
It might be representative of greater society: when people interact with ChatGPT, they treat it as an oracle. I see it all the time on r/chatgpt - people say, "I asked ChatGPT about my long lost grandparent whose life I am researching. Apparently my grandma was a pilot!"
Seems like the lawyers are shocked the oracle lied but still think they were right to consult it.
Oh they get it. ChatGPT explained it to them, and then suggested that they take this position because it was the least bad option.
Like editing Wikipedia then citing the contents. Impossible to defend in good faith, but passes as a legit excuse in a public court.
Some people really know how to use public naivety to enforce the public’s powerlessness.
https://en.m.wikipedia.org/wiki/Wikipedia:Citing_Wikipedia
Sure, we all do it informally, but implicit in that is that you should check the real sources if you care about accuracy.
I don't see any proof that shows they intended to deceive rather than that they were clueless.
The attorney used it as a research tool, and even foolishly thought to use it to validate that it wasn't making shit up. It's a really embarrassing mistake that's only obvious in the eyes of people familiar with the technology. There is nothing about this that suggests they intended to deceive, only intent to be a lazy idiot.
- "A man sued Avianca Airline – his lawyer used ChatGPT" (27 days ago, 139 comments): https://news.ycombinator.com/item?id=36095352
- "Lawyer who used ChatGPT faces penalty for made up citations" (15 days ago, 128 comments): https://news.ycombinator.com/item?id=36242462
- "Lawyers blame ChatGPT for tricking them into citing bogus case law" (14 days ago, 175 comments): https://news.ycombinator.com/item?id=36257545
Original court documents: https://www.courtlistener.com/docket/63107798/mata-v-avianca...
The full document from the federal judge on sanctions: https://storage.courtlistener.com/recap/gov.uscourts.nysd.57...
One of many highlights: dismissing the lawyer's bad-faith argument with a footnote quoting Alice in Wonderland (footnote 12, page 18).
I have a friend who used to be a federal judge for immigration, and he said that immigration lawyers are especially the worst, because families need their services desperately, and these lawyers will take their money and spend NO time on their cases. They show up to court and are figuring stuff out real time.
He tore one lawyer a new asshole for doing exactly this for his client and ordered them to come back better prepared.
So the idea that some of them will use ChatGPT to do their work doesn't surprise me at all.
Sure, I see plenty of uses for ChatGPT in their field, but to not even check the output before submitting it is what blows my mind.
But also, pretty sure Doctors and Lawyers are among the least trusted/highest corruption professions. All that schooling is a barrier to entry to keep outsiders away, it doesnt guarantee competence. Basically once you get in, graduation rates are close to 100%.
My wife runs the resident eduction for her division and they are very aware of the dangers of sending an incompetent doctor out into the world to practice. They’ll force them through extra training and make them repeat residency if they have to.
Residents also drop out of their programs and move into other ones. It’s rare for them to drop out completely because they’d still be on the hook for $200k+ in loans. But medicine is a big field and it’s rare that someone who made it into and through med school to not be able to find something they are competent and capable at. Maybe you’re not great with patients, but you can look through a microscope all day as a pathologist etc…
Incompetent and corrupt doctors still make it through the process of course, but far far fewer than in our profession and just about any other profession I can think of.
As for lawyers:
Law schools have around a 77% graduation rate nation wide. And about 90% of law school graduates eventually pass the bar exam.
So we’re look at somewhere around 70% of people making it through after a pretty selective filter to begin with.
n=1 anecdote here:
When I was in research track biochem and bumped into premed students, nearly the entire population of premed didn't seem to care about the science behind what they were learning. They formed a very different social clique. The "top students" in that set were sharing previous years' organic chemistry exams and none of them would read papers or get involved in research. They were entirely disinterested in the science.
In my interactions with general practitioners, I like to talk about medicines' method of actions, pharmacology, the actual biochemistry behind diseases. Most of them seem to have no clue or retention of this information. I'm not trying to challenge them, either -- I'm generally curious to learn.
That's not to say all doctors are like that. A lot of the surgeons and specialists I know are walking tomes of information.
Once you get to medical research groups that formulate those protocols, that’s where research becomes important.
Isnt this even more necessary for them to learn the fundementals? Especially if Physicians claim that they are both Art and Science, which is why we can't merely use science for diagnosis and treatment.
So they do learn fundamentals but fundamentals do not come out of research papers.
When I took non-required theoretical classes, I tended to be one of the only undergrads in the class
Also most other countries have national education programs for secondary school. High school in the US is regulated at the state and local level, so it’s very variable. As a consequence, the first 2 years of college tends towards general and education.
Another note, there’s also a med school entrance exam in the US.
Agreed regarding the US high school standards but students that take AP courses in high school can take the MCAT. They may not do the “best” but they will know some of the material. Schools in the US like UMKC have 6 year programs were high school students take the MCAT.
That’s a very different sample. Then you have to consider that the people that you are most likely to notice are the most vocal self identified “premed” students since most colleges don’t actually have premed degrees.
As far as interactions with doctors. GPs are the least likely to get into specifics because of the generalist nature of their work. Expecting them to remember something that they haven’t used in 10+ years is asking a lot.
Understanding calculus is incredibly useful to software engineers. But try asking a practicing software engineer 10 years out of school some textbook calculus questions.
https://www.opensecrets.org/federal-lobbying/top-spenders?cy...
They are the 4th largest briber in the US.
Finding a difference between bribery and federal lobbying with money, is the same outcome.
Money goes from person who wants, to person with power who receives. Everything else is theatre.
The money that comes from lobbying is also limited and pays for reelection, the candidate doesn’t get to use it for personal expenses.
There are clearly problems with the system, but yelling no! it’s exactly the same as taking bribes isn’t helpful.
Lobbying is legal because we live in a society that places such a high value on money that we've accepted bribery as a means to increase its power.
Also I wonder how well the training data was cleaned from all potentially harmful stuff like for example the sovereign citizen ideas...
You really need not wait that far if you have been observant enough, they are they were at one point clownish enough to believe in religion, now the vast majority of them believe in 'science' etc. Over reliance on Chat GPT is just the one of the newest manifestation of laziness and dumbness.
Did they risk it? Even after it being obvious the lawyer was lying and trying to cover up, and not acquiescing to their wrongdoing even after all of it, all they had to pay was $5k.
They still risk being disbarred (not just hypothetically, it might actually happen) + taking a massive hit to the reputation (their own + the firm). The latter one alone is a strong punishment, given how widely-publicized this incident was in the media. It cost the firm a lot of potential clients, and the lawyer might struggle with finding a decent position after the fact (given how much more reputation-based the legal field seems to be compared to something like engineering).
With that information I think it's less mind-blowing, they were simply assuming ChatGPT could do the work of a paralegal
Of course after some number of iterations, AI may gain the ability to self-check and self-correct to the point of achieving 100% accuracy, but I think that milestone will quickly lead to AGI and job security will quickly become an obsolete topic for every industry anyways.
It's because, just like human memory, they aren't databases or search engines. Generative transformer models are basically next-word-prediction machines on steroids. They take the input and try to "guess" the most likely reply based on their training data.
These machines have no way to distinguish facts from fiction, only probabilities of combinations of words that would make the most plausible reply.
> Is it possible to prevent it?
There are methods to prevent this by incorporating specialised knowledge databases into the training material of these models. This, however, only works with models that have been finetuned on very specific tasks and topics [1].
Other approaches use AI to transform human questions ("bag of words" inputs) to queries into structured knowledge bases, match the results (e.g. tree-like structures of context and facts) to the question and turn them back into human language [2]. The downside of these methods is that they're currently limited to simple QA formats and won't feel as "natural" as talking to chatbot and requires carefully prepared and curated knowledge databases.
[1] http://jens-lehmann.org/files/2019/iswc_bert_simple_question...
This reminds me. Whatever happened to IBM’s Watson? It’s not a LLM right? But it seem similar enough function wise.
If you mean that some lawyers use paralegals to do word searches for potentially-relevant case law (i.e., published judicial opinions as precedents), that might be true. But in my experience, good lawyers either do the raw research themselves, or at a minimum they use junior lawyers to do the initial searching for precedents; then if necessary, they do supplemental searching themselves, based on their greater experience.
Now many, if not likely most, of the well-schooled are generally intelligent, but of course there are also quite a few rich idiots as well.
(Today, of course, most of the latter go into tech.)
This is why the law firm's contention that they were acting in good faith is laughable.
When the tool itself shows a prominent disclaimer that it "May occasionally generate incorrect information," then the law firm can't reasonably argue that "We made a good-faith mistake in failing to believe that a piece of technology could be making up cases out of whole cloth."
It was either gross negligence or intentional malfeasance.
The most interesting thing to me is why the firm is continuing to try to excuse its actions, rather than taking full responsibility for them. I guess they assume that the classic Shaggy "It Wasn't Me" defense is less likely to lose them business, because they think so little of the clients they're trying to attract that they're wagering they'll buy it.
The researchers training the ones we use now are in a massive hurry. They aren't meticulously curating datasets, checking usage rights, hiring a huge field of human curators, or thoroughly field testing them. They are madly working on a new improvement, publishing it in a paper, then moving onto the next thing.
This has been a cultural issue in AI land for years. These experimental models were never meant for the general public, yet here we are.
Basically, if this paper is right, 2021 was the last year usable training data could be pulled from the internet. It could be all downhill from hereon out, as the amount of AI generated content will soon overtake actual human generated and -curated data on the internet.
So the party could be over soon ¯\_(ツ)_/¯
Then when a judge told them "hey, we can't find any of these cases, what the hell is going on?" they doubled down. They're basically saying "trust me bro, these rulings totally happened and they totally say we're right".
This is like your local major getting tricked into buying the Eiffel tower using public funds and then defending himself by saying he asked the guy he bought the Eiffel tower from and the guy said he was totally legit.
The good lawyer's job is not just to advocate the client's position — it's also to do "completed staff work" for the court in the process, even acknowledging any weaknesses in the client's case (which builds credibility).
As we've seen in the ChatGPT case, judges tend to respond badly when they think a lawyer is trying to put one over on them.
And conversely, judges often respond favorably when they perceive that a lawyer is being thorough and intellectually honest, not merely throwing up any bullshit argument that comes to mind. A fond memory from years ago is when a federal judge called me and my opponent — a well-known lawyer in our field — into his chambers to talk about a pending motion that involved an uncertain point of law, which we had extensively briefed. My opponent had not done an especially-great job; my own briefing had acknowledged the weak points in my client's position but also explained why my client should win anyway. The judge said, I don't really know what the answer is here. He then turned to me and said, But I'm more comfortable with your analysis. And he ruled my way – which won the case for my client (a company you might well be a customer of today).
Lawyers admitted to the bar aren't innocents. They're "officers of the court", meaning the legal system relies on them to avoid BS in their citation of other cases and of statutes and laws. It's never been acceptable for a lawyer to sign off on something without reading it and making sure it's right.
What if registered professional civil engineers started asking generative AI for sewer-system or bridge designs and then didn't inspect them before approving them and handing them off to the guys with dozers and cement trucks? Not only would people fall into rivers, but the rivers would be polluted.
The real world consequences are similarly serious when lawyers make stuff up, or let their interns or word-processing software make stuff up.
-On March 1, They filed a submission with fake citations from chatGPT.
-On April 11th court told them the cases couldnt be found and to produce full transcripts of the cases by April 25.
-On April 25, They came back and said they don't have them and couldnt find the cases, only the limited excerpts provided in "online databases".
-On May 4, the court asked them why they ought not be sanctioned for citing non-existent cases to the Court.
-On May 25, They said we found these cases on ChatGPT and thought they were real.
I guess the more transparent response on April 25 would have been to say "We weren't able to find the full cases and we now believe they probably dont exist ". Im not sure if the latter part is material if they already admitted they cant find the cases. If you cant produce the cases, is the reason relevant at that point?
Just like Wikipedia, ChatGPT is great for generating leads, but you need to check the actual primary sources! How do people not get this?
It is mentioned also in the article:
>then continued to stand by the fake opinions after judicial orders called their existence into question.”
> then continued to stand by the fake opinions after judicial orders called their existence into question.
At first it was just generated citations. The opposing lawyers and judges could not find the cases, and so they then asked chatGPT for them, and it generated them, and they were submitted to the court. (One of the lawyers notarized the other’s filings.)
There are even transcripts of chatGPT telling the accused that it is not an authoritative or up-to-date legal database.
—-
The punishment could arguably be more severe, since it demonstrated that these lawyers do not read their cases, particularly the one that notarized everything.
I just can't agree. To my way of thinking there are very few (if any) things that should be "career ending" the first time they happen. And the main guy involved here, as I recall, has a long career with what has been a spotless record until now. I just can't see the justification for ending that career over one fuck-up.
But, again... the punishment does seem lighter than I expected. I was thinking that any fine associated with this should be enough to hurt a bit. Something like $250,000 or something. This amounts to a slap on the wrist.
Just to clarify my position here: I do think this is a major fuck-up. Like, really major. To the extent that I think the punishment should not be "career ending" it's just a reflection of my seeing "career ending" punishments as something of a "nuclear option" that should be avoided in all but the absolute most extreme cases. I guess I would say that if punishments spanned a scale from 1-10 with "10" being "disbarment" then this should probably be about an 8 or 9 in my mind.
These people should not be practicing law.
That, combined with the lawyers involved having decades-long clean slates, as well as this case being outside the normal realm this team would operate in, as well as ChatGPT being relatively new at the time this happened makes it all seem pretty proportional.
Maybe you could say they deserve worse for how they handled things after they got caught, but I struggle to really have a problem with this punishment considering the records of those involved.
Both the fine and the public shaming are very light "punishments". Everyone's pointing and laughing now, but in a year nobody will even remember their name and their career will be fine.
So does Google, and anyone researching these lawyers or this firm in the future...
The fact that they did no research on ChatGPT, and they didn't verify the output from ChatGPT, is pretty bad.
They're being punished more so on how they used their resources, not for using ChatGPT.
Yes, exactly. If they'd used a human intern instead of ChatGPT, it wouldn't change the situation at all.
The client may want to sue them and seek damages or something and that'd be fair but I don't think their career should end. That's harsh.
These twits should be disbarred, and without hesitation.
This presupposes that the use of ChatGPT is the reason the case was lost. In this case, the judge found the mistake, called it out, and fined the lawyers for their carelessness. It doesn't necessarily follow that p1) ChatGPT was used, p2) Case was lost, c) Case would have been won if ChatGPT was not used.
Or to waste a lot of their clients money on cases that they should have known could not be won from the outset, as is the case here.
It's not equivalent to that. Nobody lost their leg over this.
Stupid mistakes are usually not career ending, fraud and cover up usually is.
Because certain careers, e.g. lawyers, surgeons, and law enforcement give their practitioners the ability to make or break their clients, potentially killing them or ruining them financially and/or socially. Such professions must, for reasons of social sanity, be held to higher standards than someone who's career is, e.g., writing web apps or serving coffee.
I'd wager a guess that this will only get worse over time. LLMs will do more stuff for people - from helping school children to get through their homework, to higher learning assignments and all the way up to the "real" jobs. Along the way people might just place far too much trust in these tools. I'm pretty sure that's something we can all agree on.
What I do find striking here though, is the incredible laziness of the lawyers. Not even checking whether or not the cases actually happened? That's just bad lawyering. The AI actually helped the system to call them out on their bad lawyering.
That's not "bad things happening because AI" but just plain stupidity.
This thread: do they not know what ChatGPT is? Did they not verify the output? How could they believe this?
Just maybe when an army of grifters jumps on the next hottest AI thing to advance their careers that kind of bald-faced lying has consequences?
Because if you can't, it's not the fault of "grifters"...
Given the marketing and the way Microsoft and Google were tripping over themselves to "catch up", the impression in the media and to the general public is that chat bots are a magic silver bullet for everything. Even a lot of tech people are unclear about exactly how these things work.
The case seems pretty clear: "They said it was artificial intelligence, but it wasn't very intelligent to make up cases. Look at the damages it's caused me..."
As entertaining as this is to ponder, I don’t think they’re that clever. I don’t think they’d do so much damage to their reputation for such a flimsy case. They’re just incompetent.
I went back into my texts since I had recommended them chatGPT. I screenshotted the part where I said:
"You can't trust it for valid information"
I even told him that everyone needs to be punished by incorrect information from ChatGPT once to really understand what its capable of. You need that traumatic event.
Gosh, why can't people just listen and accept other people's thoughts. Why must we learn individually?
Because other people's thoughts are usually wrong so you cant trust them.
Why can't people just do research the real way rather than circuitously go to the wrong answer machine and then double check everything it says?
If you've evolved from a species that is profoundly stupid, as a rule, then perhaps an adaptation/mutation that makes you mistrusting of that species advice is one that tends to propagate.
For every "don't use ChatGPT, it's unsafe", there's a "I talk to magical beings who tell me you're not allowed to eat good food and you should mutilate your genitals".
Statistically, refusing to learn from other people is probably the best bet.
I wonder when tech firms will actually be held responsible for false advertisement.