Judge Learns Both Sides Used AI, Cancels Trial, Kicks Everyone Off the Case | Hacker News Reader

Judge Learns Both Sides Used AI, Cancels Trial, Kicks Everyone Off the Case

404media.co

97 pointsarto22 comments

Wow, this is amazingly bad. My favorite excerpt:

> “The Court is compelled to note that it has serious concerns that Wilson has continued this practice of AI misuse in other cases after she was put on notice of her violations in this case,” she added, noting that other judges in other cases had found hallucinated cases in Wilson’s filings as recently as April, four months after she was initially asked to explain her AI use in this case. “Her continued AI misuse demonstrates an extreme dereliction of professional responsibility on her part. Though this Court cannot consider subsequent conduct that did not occur before it in determination of the appropriate sanction(s) in this case, it finds that at minimum Wilson’s apologies to this Court on January 20, 2026 were not sincere.”

On the plus side, seeing behavior like this makes me realize that I can be impactful even in areas that I have little experience. The bar (ha!) is really quite low.

The pure audacity to get caught and chewed out by a judge and then continue not reviewing briefings for hallucinated cases.

It is way harder to review that sort of stuff then create it correctly by yourself.

If their they expect tp speed up and rely on reviewing, it is pretty much guaranteed some lies will end up in documents.

Is it really though? Now admittedly I’ve never worked with LexisNexis or Westlaw, but would it really be that difficult to have a tool just check if the citations actually exist?

Westlaw is very expensive

You also have to read them and check whether they claim what the document claims. Just checking whether it goes somewhere is not enough.

This whole thing is requiring people to do exactly the thing they are bad at - controling output of something else while being passive and not loosing attention.

Opposing counsel is also being called out by the judiciary for not recognizing that multiple citations counsel has put forth are not only not real, but actually counter to existing law.

Link to the official document - https://documents.lastweekinlaw.com/view/gov.uscourts.msnd.5...

This problem mostly goes away the minute lawyers lose their license for this shit. The system was designed with a way to kick out bad lawyers and keep those in the profession acting ethically, but for some reason getting a state bar association to use that system is next to impossible.

Until judges start imprisoning lawyers for contempt, this will continue to happen.

https://archive.ph/My7GF

"Notably, that tool was described as being built for “in-house legal research,” and that the tool in question is not supposed to hallucinate cases."

Heh, nice try.

That part actually seems possible, and I’m surprised it didn’t work. Last I looked, legal citations had an exacting format I would think you could parse out and check if the case exists at least. Won’t work if the AI hallucinates the citation format though, I guess.

You could maybe even verify quotes if the citation was attached (treat ellipsis as a regex “.*” and check the case text). That part would probably be imperfect, but I’d think useful at least.

feels like an onion headline. crazy times

I think that this says even more about the law than about lawyers. The law seems to require a lot of documents that nobody actually wishes to read -- exactly the kind of thing that we like to turn over to AI. The lawyers don't read the briefs they generate, and they're expecting the judge to only skim it.

The entire thing feels like it should be condensed down into a completely different format. We're doing the law the way it was designed a thousand years ago.

I hate to be the "Why does your field have a whole journal anyway?" guy from XKCD, but I feel like AI is pointing out a problem.

Frequently, it seems like we should turn some processes over to AI, then shut the AI off and see what, if anything, is actually lost. What do the lawyers here actually want the judge to know? What can be done to ease the work on both sets of lawyers and the judge by drilling down to the actual information hidden within the LLM-generated verbosity?

> I think that this says even more about the law than about lawyers. The law seems to require a lot of documents that nobody actually wishes to read -- exactly the kind of thing that we like to turn over to AI.

Why would you turn them over to AI? If nobody is going to read a document, it doesn't need to exist in the first place. Fortunately judges do read this stuff which is how they keep finding out that it's filled with AI invented bullshit. These lawyers are just trying to get out of doing their job by having AI spew out legal looking garbage while billing their clients for time they spent not working.

This is why we have to ban use of legal AI by non-lawyers!

Being able to think of a hypothetical motivation is not evidence that an action is due to that motivation or that the action is necessarily even a bad idea. There's a lot of smart people here, lazy and specious arguments likely don't go far.

The article was about misuse of generative language models by lawyers.

It sounds like lawyers are just as likely to use this stuff stupidly as anyone else

Everyone's dumb, just in different ways.

lawyers should appreciate how tiny, often imperceptible errors, or changes to a document can have serious implications for the life and finances of the people they represent.

plus there are already well established automated document review tools. I had these tools and their firms as data center customers as far back as 2012.

the difference is that the tools flag or explain things, while the LLMs in question here generate wording.

Reply on news.ycombinator.com