Hacker News Reader

Comment by shalalalaw | Hacker News Reader

We're a very tech forward law firm, and we're bullish on AI. The issue is that lawyers are traditionally tech illiterate, and they treat Gen AI like a search engine that puts results in narrative form. Realistically, I think AI generated motions and contracts are the future, and this instance will be looked at by every tech averse lawyer to try and stymie progress in the field. These lawyers deserve their sanctions for being so reckless with things they don't understand, but rather than take away that lawyers need to learn tech, lawyers will say tech is bad. I almost wish this was non-news so it wouldn't further push the legal industry into the past, but those clients were wronged and I guess people need to know what to beware of when hiring a lawyer.

Personally, we get really good results from using AI, it's already present in all of our processes, but we tell it what to generate, rather than rely on it to know better.

View on HN

It's not just lawyers who think that ChatGPT is a search engine. I've observed this many times in my vicinity, people from all walks of life think that Star Trek is here and computers now respond accurately to natural language queries. For non-techies, "just asking the computer" is so much more convenient than translating your question into traditional search queries.

So I guarantee you that stuff like this is happening daily across all industries. Depending on the profession, people will lose money or get hurt as a result of someone blindly trusting this technology. I can't prove it, but statistically, that's basically a certainty.

In my opinion, you can't overstate the importance of articles like this, which point out the limitations and highlight the dangers. I'm also against banning. But lay people need to be informed what ChatGPT is and is not, and OpenAI won't do it because they want to ride the hype train.

In local-to-me politics we have a report on changing the admissions process for specialty programs in the public school system - which had a bunch of fake citations and people suspect was written with the "aid" of ChatGPT.

https://www.thestar.com/news/gta/2023/05/26/tdsb-fires-resea...

> It's not just lawyers who think that ChatGPT is a search engine.

Let’s not forget that Google often puts incorrect information in their snippets/factboxes or whatever they call them.

Computer, what's the formula for transparent aluminum? Seriously, I got ChatGPT to spit out a scientific-seeming paper on the formula and manufacturing process for transparent aluminum. It did note that there's a real thing, aluminum oxynitride, which is the closest thing we have to the Star Trek material. It even wrote the following abstract, based on my prompt:

> This scientific description provides an overview of the formula and manufacturing process of transparent aluminum, a material used in applications where both structural strength and transparency are required. Transparent aluminum finds extensive use in diverse fields, including public aquaria, where it allows for the display of large marine organisms. The description outlines the chemical composition, key properties, and the manufacturing steps involved in creating transparent aluminum.

Whether or not the six-step manufacturing process it came up with is correct or not, I haven't the expertise to say.

> Whether or not the six-step manufacturing process it came up with is correct or not, I haven't the expertise to say.

I mean, you can try it and perhaps get rich. Or, probably more likely, die in a horrible explosion.

[the material is] placed in a high-pressure vessel, and temperature and pressure are simultaneously increased to near the melting point of aluminum.

I don't need to try that to know that molten metal under high pressure at over 1000C is something I'm not ready to deal with.

Idk it's pretty reliable even now if you chuck a vector db of "knowledge" in and inform the GPT in the overall prompt that it is must not go outside the bounds of knowledge provided with the user's query, and that if no hard knowledge is provided that it should respond along the lines of "I don't know" (or search the internet and parse results for them).

I imagine at some point this behaviour will be built in. People are treating GPTs like they're knowledge bases, but they're not. The fact that they can answer simple or complex questions correctly is only a byproduct of the transformer being taught how to string language together.

It's like an artist who learns to draw by making paintings of raccoons. You can't then ask them "what do raccoons like to eat?" or "what foods are poisonous to a raccoon?" just because they learnt to draw by painting raccoons. This is how people are treating GPTs atm. They believe in it because they ask the artist "what colour fur do raccoons have?" and because the artist can answer that correctly, they assume all other answers are factual.

> [L]awyers are traditionally tech illiterate

This is the main reason I think disbarment as the punishment in this specific instance may not be fair. There are people who are unaware of the limitations of these systems and the risk of these confabulations occurring.

While I don’t think disbarment is inappropriate, I would rather see the New York State Bar use this to require some better understanding of these emergent technologies or even better have all the State Bars start discussing some standardized training about this because it’s easy to see a person trying to treat this as LexisNexis.

Is disbarment about fairness? Is the primary goal of such proceedings to rehabilitate and apply a sort of justice?

Certainly, civil and criminal courts have those as their raison d'être. But I thought licensing boards had an entirely different purpose. If I surgeon was a good guy who genuinely wanted to help people and who didn't engage in any sort of malfeasance... but even so, he just kept slicing aortas open accidentally through incompetence, the board should say "aw shucks, he's had some bad luck but he really wants to heal people".

This is the same. The court system is replete with circumstances where a client does not get a second chance at pursuing justice. A lawyer that fucks that up, even if doing so in good faith, leaves them with zero remedies. This might have been a bullshit "Slippin' Jimmy" case this time, but the stakes could've easily been higher.

I don't think I want to live in a world where fairness plays any part in the decision by the bar on this matter.

Disbarment is usually considered a punishment of last resort. That the failure of the attorney to carry out their obligations is so absolute, that it justifies taking away their right to practice law in a given state. There are certainly other measures that can be done here that are of a similar rebuke, just not as final. A suspension or temporary disbarment is also possible.

We don’t know the full situation here, but a personal injury case against a bankrupt airline for striking someone in the knee with the serving cart seems remediable?

Disbarment usually happens in cases where attorneys fail to file timely repeatedly at the expense of their clients and after multiple admonishments to stop that; utterly fail in their fiduciary obligations (i.e. they were acting like an escrow and then instead gambled the money away in Vegas).

> that it justifies taking away their right to practice law in a given state.

This seems a little weird. As far as I understand it, no one has a right to practice law.

There is a privilege that can be acquired, it one meets the requirements. If you somehow got through without meeting those, or if you start to fail to meet those... time for a new career.

> We don’t know the full situation here, but a personal injury case against a bankrupt airline for striking someone in the knee with the serving cart seems remediable?

I don't know about this particular case, but many cases and circumstances can be a "one shot at it" scenario. You fuck it up, it's tossed and you can't refile. There are many reasons and details, any of which might be messed up by a lawyer relying on a silly chat program to draft motions. One might miss an absolute deadline. It might be dismissed with prejudice. Appeals might be exhausted. This could even be true of the case in question.

In some cases, it might even be true if it was a criminal trial and your defense attorney was incompetent, that you don't get a chance to appeal. In California, I think, those are Marsden cases (someone correct me if I'm wrong). For those, you have to raise an objection during the trial.

So, if someone found out that ChatGPT gave their lawyer bad advice the day after their conviction... well, oops. No appeal for you.

I'll say it again. I do not want to live in a world where law license proceedings are decided on a "what's fair to the bad lawyer" basis. No one has a right to be a lawyer, if you're bad at it there are plenty of other occupations you might make a living with where incompetence doesn't threaten so many lives and livelihoods.

Let's change the professions a little bit.

Imagine getting fired and barred from writing code ever again over a bug you introduced because you used copilot and didn't spot the issue.

Pretty sure that would be considered an unacceptable infringement of basic human rights here.

You can assert your ideals all you want but the fact is that professions that govern themselves invariably end up with "what's fair to the bad lawyer".

Licensed professionals are licensed (should be, there are notorious exceptions) because if those professions remain unlicensed, horrible things happen.

Lawyers and medical doctors are two of those. Yes, it would be wrong to prohibit the Starbucks barista from making coffees, no matter how many times such a person burned it.

Software engineering probably falls between licensed professional and burgerflipper on that scale... but let's not full ourselves. If you were working on firmware for medical equipment, then yes banning you from ever doing it again because you used ChatGPT when making a heart rate monitor is just and fair.

Not all of our software matters. But the people working on code for space vessels or aircraft or as in my example, medical equipment? I'm more than happy to see them banned from these things for life if they were to do that.

> You can assert your ideals all you want but the fact is that professions that govern themselves invariably end up with "what's fair to the bad lawyer".

This is irrelevant. We're all aware of how underperformant oversight tends to be. The point is to fix that, to rally against its eventual decline. Certainly I don't know why anyone would want to embrace your attitude of defeat/acceptance.

If your doctor asked ChatGPT to tell him how to remove your appendix, followed the directions, and subsequently removed a kidney instead, would you want him to lose his medical license?

For sure, but the difference there is that someone was actually severely wronged. The worst that happened here was some people had time wasted.

I think a punishment where the lawyer had to pay for all the time he wasted for the judges and various legal clerks (and his client) would be sufficient personally.

He is unlikely to make the same kind of mistake again I would think.

> The worst that happened here was some people had time wasted.

At least one party to the suit, if not both, are going to end up spending extra money. Plus it wasted public resources – the time of the judge and court staff, and their salaries, and more – and cost taxpayer money. Your remedy of having the lawyer pay might bankrupt him, and it doesn't really make whole the other party. In cases where one party has limited financial resources or perhaps is close to death (thing capital punishment, or malpractice), this isn't just waste. Someone could be severely wronged.

that is not a meaningful difference when the issue is that both professions can use tools but are responsible for the results of using them, and thus obligated to apply their professional judgement before and after using a tool that can hurt people (in this case, their clients, at least)

Out of curiosity, why do you think AI generated contracts are the future? Do you draw a distinction between contracts generated by AI and, say, contracts “generated” by first-year associates (i.e., using precedent to generate a first draft appropriate for the deal that’s then iterated by more experienced lawyers)?

Also, how is this incident going to push the legal industry further into the past? Do you think lawyers are going to, like, stop using email because of this?

... retrieval augmented search is here today and is available in ChatGPT with plugins or integrations with vectorDB systems. A lot of AI systems are "search engines that give you narrative outputs"

You sound like an AI generated post.

>ai generated contracts are the future.

Man, why do you guys get paid so much? Contracts need semantic correctness, ie the verification of the logical consequences of natural language. This is an AGI determining problem. At the point this exists, humans are basically obsolete as workers, and you don't have to worry about your law firm keeping up anymore.

Let me get this straight, you're saying if AI can be a good lawyer AI can do anything humans can do?

So you're trying to say that lawyering is the hardest job in the world and only the smartest people can do it...?

I claimed that it was a task that most people can do, but is really genuinely, intelligent. I actually thought it was everyone, but then you responded with your comment.

Contracts have lots of standard clauses or near standard variations of such, and as a consequence plenty of "dumb" template based generators already exists, and you may come across contracts that have just the occasional line manually written.

At least one company already integrates or is about to (not sure if it's in production yet) LLMs in theirs to do effectively smarter completions.

It won't be fully automated any time soon, but it will certainly eat into a lot of the simpler work.

Reply on news.ycombinator.com