The only problems are it could be convincingly wrong about anything it tells you and isn't liable for its mistakes.
If a LLM can pass the bar, and has a corpus of legal work instantly accessible, what prevents the deployment of the LLM (or other AI structure) to provide legitimate legal services?
If the AI is providing legal services, how do we assign responsibility for the work (to the AI, or to its owner)? How to insure the work for Errors and Omissions?
More practically, if willing to take on responsibility for yourself, is the use of AI going to save you money?
The law, which you can bet will be used with full force to prevent such systems from upsetting the (obscenely profitable) status quo.
Basically, add a "validate" step. So, you'd first chat with the LLM, create conclusions, then vet those conclusions with an expert specially trained to be skeptical of LLM generated content.
I would be shocked if there aren't law agencies that aren't already doing something exactly like this.
They'd be sued out of existence.
"In terms of being convincingly wrong, it's not like lawyers never make mistakes..."
They have malpractice insurance, they can potentially defend their position if later sued, and most importantly they have the benefit of appeal to authority image/perception.
I guess you'd have to have some way of knowing that the "malpractice insurance ID" that the GPT gave you at the start of the session was in fact valid, and with an insurance company that had the resources to actually cover if needed...
In the tests they are shown to be pretty close. The point I made wasn't about more mistakes, but about other factors influencing liability and how it would be worse for AI than humans at this point.
This is the key point. Even if assume the AI won't get better, the liability and insurance premiums will likely become similar in very near future. There is a clear business opportunity that's there in insuring AI lawyer.