Hacker News Reader

Comment by boomlinde | Hacker News Reader

> I don't think my comment implied that any testing and validation process will be fool proof.

It does by, attributing a change in quality to a "testing and validation problem".

If there is a change in programming methodology, as far as we know no change in testing and validation methodology and at the same time an increase in the production of bugs, it's not reasonable to attribute that increased error rate to testing and validation getting worse.

It's like adding spinning saw blades to the fronts of cars and then attributing the increased fatality rates to pedestrians not wearing safety reflectors.

View on HN

If it happened as you describe, with no change in the testing and validation processes, then it exposed an existing weakness in those processes, which should be improved.

In your analogy, what I would say is that the saw blades on the front of cars situation exposed a weakness in the regulatory regime for what can be put on the front of cars, which should be improved.

> If it happened as you describe, with no change in the testing and validation processes, then it exposed an existing weakness in those processes, which should be improved.

I remind you that the belief you are defending is that an increase in the number of bugs produced per some unit of implementation "implies that testing and validation has gotten worse", not the general idea that you can or should improve software quality by improving testing and validation.

If we adopt the realistic perspective that the testing and validation process can only catch some fraction of bugs, then an increase in the number of bugs produced in the implementation stage is bad irrespective of what that fraction is. You can improve the testing and validation process so as to minimize the fraction of bugs that get past QA, but when you multiply that by the number of bugs produced in implementation you will be worse off if you produce 7 bugs per month during implementation than if you produce 5 bugs, regardless of whether you can also improve the testing and validation process. You will on average, over time, have fewer bugs in production if you produce fewer bugs during implementation given any testing and validation process that isn't completely fool proof.

No, what I'm defending is my initial comment in this thread:

> Isn't this a testing and validation problem?

One way for it to be that kind of problem is by exposing weakness in the existing approach. Another way would be if the approach was worsened. But both things fit my original contention.

I don't disagree with your concluding paragraph at all. But what I think is that there is no reason that implementing with llm based tooling implies an increased defect rate. Rather, I think it implies that there are existing or new holes in the quality process. That doesn't imply that I think a defect rate of zero is possible. But I think holding the defect rate steady is possible.

You could also just remove the product from the market. That will guarantee a reduction of bugs in production to 0. That exposes a weakness in the existing approach to marketing and overall product design, meaning the problem is in marketing and product design.

... I guess? I don't really understand your point.

Reply on news.ycombinator.com