Surely there is a mathematical model here, but intuitively I do think there is an infinite number of typos and errors you could contain in the set of finite books, and similarly there would be an unlimited number of bugs and vulns in the set of Turing machines.

> There's only a finite number of bugs and vulnerabilities.

The context of an LLM is also finite.

Vulnerabilities are perpetually being created, and this will be true no matter how good LLMs become at writing code - there's simply too many factors that can contribute to something apparently benign becoming dangerous.

Semi agreed but I think that we are likely to see a ton of vulnerabilities found in the near term as AI's go through codebases and find all the stuff that was missed over the years. Once that period has (mostly) passed I imagine things will slowdown to somewhat similar to a normal stream of bugs and vulns and as new code is created.

something about the current trend of AI vulnerabilities reminds me of the old Asimov Robot series and the three laws of robotics:

https://en.wikipedia.org/wiki/Three_Laws_of_Robotics

I doubt you can prove that.