Ask HN: How to Measure Software Quality?
I cannot find a way to find a good metric for software quality...
Number of bugs found by the customers / per month? Any other ideas? What are the quality KPIs in the companies you're working for?
Number of bugs found by the customers / per month? Any other ideas? What are the quality KPIs in the companies you're working for?
Poor design quality increases the chance of poor build quality. That's why it's important to measure both.
As an aside, I've always felt that 'Defect' is a more useful term than 'Bug'. 'Bug' seems to be too open to interpretation (i.e. one persons bug is another persons feature). 'Defect' however, has a useful definition:
'An operation is defective if it doesn't conform to the specification.'
This provides a solid basis for identifying Defects. Once you can formally and accurately establish what a Defect (i.e. Bug) is, you can use the following two simple formulas as KPIs.
1. Design Quality Change Requests/Requirement Specification (DQ = CR/RS) 2. Build Quality Defect Reports/Requirement Specification (BQ = DR/RS)
The ideal is zero. In other words, zero Change Requests per Requirement Specification and zero Defect Reports per Requirement Specification. However, I suspect that no project in history has ever achieved that. Nevertheless, it's the quality target to aim for. Once you start treating these values as KPIs you can start to monitor the two numbers over time and steadily work to reduce them, thus improving quality in a measurable way. For instance, you may find that the project you're working on has a DQ of 2.5 and a BQ of 3.6. Your mission (if you choose to accept it) is to steadily increase quality so that those two numbers reduce over time.
By viewing the results across a date rand you can start to see quality trends.
Design Quality is an important metric to monitor because that's often where the weakness lies in software projects. Managers love to offload design to developers in an ad-hoc way, largely because design is hard and time consuming (and no-one likes to write specs).
If you're concerned about build quality, first check the design quality. If you're building a product from poor, incomplete, inaccurate specifications, you're going to find it much harder to achieve decent build quality.
Goodhart’s curse: Every metric that can be tracked gets incentivized and ultimately gamed.
Beware what you choose to measure.