amount of data you'd need to learn to generate and cite fake court cases and give advice that may or not be correct with equal apparent confidence in both cases
fixed that for you
Tool using big LLMs when asked can double-check their shit just like "real" lawyers.
As the confidence of advice, how much the rates of the mistakes are different between human lawyers and the latest GPT?
Notice I am not talking about "rates of mistakes" (i.e. accuracy). I am talking about how confident they are depending on whether they know something.
It's a fair point that unfortunately many humans sound just as confident regardless of their knowledge, but "good" experts (lawyers or otherwise) are capable of saying "I don't know (let me check)", a feature LLMs still struggle with.
IMHO, that's irrelevant. People don't really know they level of confidence either.
> feature LLMs still struggle with.
Even small LLMs are capable of doing that decently.