How does he solve the Fruit of the Poison Tree problem? For all he know, his LLMs included a bunch copyrighted or patented code throughout the codebase. How is he going to convince serious people that this port is not just a transformation of an _asset_ into a _liability_?
And you might say that this is a hypothetical problem, one that is not practically occurring. Well, we had a similar problem like this in the recent past, that LLMs are close to _making actual_. When it comes to software patents, they were considered a _hypothetical_ problem (i.e. nobody is going to bother suing you unless you were so big that violating a patent was a near certainty). We were instructed (at pretty much all jobs), to never read patents, so that we cannot incriminate ourselves in the discovery process.
That is going to change soon (within a year). I have friend, whom I won't name, who is working on a project, using LLMs, to discover whether software (open source and proprietary) is likely to be violating a software patent from a patent database. And it is designed to be used, not by programmers, but by law firms, patent attorneys, etc. Even though it is not marketed this way, it is essentially a target acquisition system for use by patent trolls. It is hard for me to tell if this means that we will have to keep ignoring patents for that plausible deniability, or if this means that we will have to become hyper informed about all patents. I suppose, we can just subscribe to the patent-agent, and hope that it guides the other coding agents into avoiding the insertion of potentially infringing code.
(I also have a friend who built a system in 2020 that could translate between C++ and Python, and guarantee equivalent results, and code that looks human-written. This was a very impressive achievement, especially because of how it guarantees the equivalence (it did not require machine-learning nor GPUs, just CPUs and some classic algorithms from the 80s). The friend informs me that they are very disheartened to see that now any toddler with a credit card can mindlessly do something similar, invalidating around a decade of unpublished research. They tell me that it will remain unpublished, and if they could go back in time, they would spend that decade extracting as much surplus from society as possible, by hook or by crook (apparently they had the means and the opportunity, but lacked the motive); we should all learn from my friend's mistake. The only people who succeed are, sadly, perversely, those who brazenly and shamelessly steal -- and make no mistake, the AI companies are built on theft. When millionaires do it, they become billionaires -- when Aaron Swartz does it, he is sentenced to federal prison. I'm not quite a pessimist yet, but it really is saddening to watch my friend go from a passionate optimist to a cold nihilist.).