Ask HN: How to measure QA in a startup?

We have a web-app and a QA engineer who's helping to ensure we don't ship outages and we know about severe regressions. (We're not aiming for pixel-perfect at the moment, but for good enough.) She's doing that by building up a Selenium suite that catches regressions just fine, spec-driven testing for new features and exploratory testing for the unexpected.

How can I give her a sense of “a job well done”? I can and do /tell/ her, cuz she's doing great, but I'd like to do better, and /show/ her: Here's how we've been doing, and this has improved in the last weeks; you're doing great.

We sat together to think about this, and came up with collecting those numbers per week:

* #severe bugs (bugs in our core functionality that are encountered by customers for whom there are no workarounds) * #reported bugs (reported by customers) * #caught severe bugs (bugs found in staging that hold up deployment — basically bugs in above “severe” category that we caught before they went live)

If those numbers are looking good, she feels she's doing a great job.

How are you doing that? Do you have better ideas to measure the success and progress of QA in a startup?