It's a logical reaction, at least superficially, to the touted capabilities of Gen AI and LLMs. But once you start trying to use the tech for actual legal applications, it doesn't do anything useful. It would be great if some mundane legal tasks could be automated away--for example negotiation of confidentiality agreements. One would think that if LLMs are capable of replacing lawyers, they could do something along those lines. But I have not seen any evidence that they can do so effectively, and I have been actively looking into it.

One of the top comments on this thread says that LLMs are going to better at summarizing contracts than generating them. I've heard this in legal tech product demos as well. I can see some utility to that--for example, automatically generating abstracts of key terms (like term, expiration, etc.) for high-level visibility. That said, I've been told by legal tech providers that LLMs don't do a great job with some basic things like total contract value.

I question how the document summarizing capabilities of LLMs will impact the way lawyers serve business organizations. Smart businesspeople already know how to read contracts. They don't need lawyers to identify / highlight basic terms. They come to lawyers for advice on close calls--situations where the contract is unclear or contradictory, or where there is a need for guidance on applying the contract in a real-world scenario and assessing risk.

Overall I'm less enthusiastic about the potential for LLMs in the legal space than I was six months ago. But I continue to keep an eye on developments and experiment with new tools. I'd love to get some feedback from others on this board who are knowledgeable.

As a side note, I'm curious if anyone knows about the impact of context window on contract interpretation a lot of contracts are quite long and have sections that are separated by a lot of text that nonetheless interact with each other for purposes of a correct interpretation.

A friend of mine is a highly ranked lawyer, a past general consul of multiple large enterprises. I sent him this paper, he played with ChatGPT-3.5 (not even GPT-4) and contract creation, he said it was 99% fine and then told me he's glad he is slowly retiring from law and is not envious of any up-and-coming lawyers entering the profession. One voice from the vertical.

People started companies and succeeded for dumber reasons. Generalized "businessing" skills and placing yourself somewhere in the space where money is made counts for much more than actually knowing anything about specific product beforehand.