I'll totally grant that thing will get better over time. But a point that I was (mostly failing) to make, is that software is discrete, NNs are continuous. No matter how buggy your program is, it's got a countable number of lines, so you can have some notion of "all the bugs" (ignoring distributed systems).

But NNs are fundamentally continuous, I don't think it even makes sense to "count" bugs. You can have a list of prompts to which the model gives unwanted output, but it's a completely different ball game compared to regular software.