For what purpose? If it's for prior art, the prior at must have been publicly available, so a private LLM wouldn't work. Perhaps I'm missing your point, though.
Something like clean-room reverse engineering could be applied. First ask a LLM to describe the problem in a way that avoids disclosing the solution, then ask an independent LLM how that problem could be solved. If LLMs can reliably produce the idea in response to the problem description, that is, after running a LLM 100 times over half show the idea (the fraction here is made up for illustration), the idea's obvious.
Would we even be able to tell if the machine reproduced the invention covered by the claims in the patent?
I (regrettably) have my name on some US software patents. I've read the patents, have intimate knowledge of the software they claim to cover, and see nearly zero relation between the patent and the covered software. If I set a skilled programmer to the task of reproducing the software components that are supposed to be covered by the patents, I guarantee that they'd fail, and fail hard.
Back before I knew about the whole "treble damage thing" (and just how terrible many-to-most software patents are) I read many software patents. I found them to offer no hints to the programmer seeking to reproduce the covered software component or system.