While this paper is clearly not without merits, it intends to be more like an excuse to make a bombastic statements about a whole profession or "industry" (perhaps to raise their visibility and try to sell something later on?). The worst part is that they have actually referenced a single preprint document as "previous art" - and that document itself is not related to contract review, but to legal reasoning in general. (A part of LegalBench is of course "interpretation", and that is built on existing contract review benchmarks, but they could've found more relevant papers). Automating legal document review has been a very active field in NLP for twenty years or so (including in QA tasks) and became a lot more active since 2017. At least e.g. Kira (and Luminance etc., none of which is LLM-based) are already used quite widely in legal departments/firms around the world. So lawyers do have practical experience in their limitations... But Kira & co. are not measuring the performance of the latest and greatest models and they do not use transparent benchmarks etc. So the benchmark results in this paper are indeed a welcome addition in terms of using LLMs. But also considering its limited scope of reviewing 10 (!) documents based on a single review playbook, they should not have written about "implications for the industry". It is very much pretentious and shows more of the lack of knowledge of the authors of the very same industry than about the future of the legal services industry.

If you're interested in the capabilities and limitations, I suggest these informative, but still light reads as well: https://kirasystems.com/science/ https://zuva.ai/blog/ https://www.atticusprojectai.org/cuad