Comment by calinet6 | Hacker News Reader

Comment by calinet6 | Hacker News Reader

Parent Full thread

QA is interesting, sure. I'm more interested in the Q part, and you don't get that just by having a role or department checking work before it hits the customer.

This article does a great job of describing the minimum level of quality allowable to scrape past certain stages of growth.

The question you should be asking is: what level of quality, built in from the beginning, intentionally and with full support, will create a product so valuable to the customer as to generate a wild feedback loop of success—not just scraping by at a minimum acceptable level. What level of quality will generate returns, not just reduce your debt to an acceptable level? And more importantly, how do you achieve it?

Quality is your value to the customer. If you think of it systemically, you won't need a QA. As Dr. Deming said, "Inspection does not improve the quality, nor guarantee quality. Inspection is too late. The quality, good or bad, is already in the product." And Harold F. Dodge: "You can not inspect quality into a product." Deming advocated instead for a holistic understanding of the factors driving quality, including management and leadership, the processes used, and continuous improvement of systems.

This is the type of discussion I always see lacking from discussions of software QA. On the manufacturing timeline, it's like we're in the 1920's. It's all very realistic, but we could be so much further along.

What you describe is QA. Testing is technically QC: Quality Control. It's only one facet of QA, though the terms are usually conflated. Most companies only have a QC department.

You're proposing a definition that is out of line with how 99.99% of people use it.

Even if your definition is more logical, this isn't how language works.

I'm not proposing anything. This is a well-known concept among quality professionals. I'm going to guess you have a different specialty.

Google "quality assurance control" and you'll find a lot more, but here's one source:

http://whatis.techtarget.com/definition/quality-control-QC

You mentioned 'level of quality'. That can be pretty difficult to quantify. In the absence of well defined metrics, a lot of thinking about QA is qualitative.

"I feel like we have a high level of quality here".

That makes conversations about QA pretty challenging. Metrics is the one area I'd LOVE to see move forward.

Heh, maybe that's because the nature of quality is qualitative rather than quantitative.

There are things that can be measured that might be correlated to quality ("bugs", sales, support, etc) but ultimately, classical quality seems to be to be a marriage of what was expected or desired with what was actually there. As a developer, my take is that quality usually stems from "developing" those expectations as much as making the relatively concrete thing to compare them too. When anyone can quantify the first half of this equation, please let me know at once.

That's a really good question, and it requires some more context.

First, you have to understand your company from a systemic viewpoint. Break it down, and understand the inputs and outputs of every part of your company. You've got your individuals, your various teams, the people they work with, the systems they build, and the product itself. You can measure the input and output of each of these systems in meaningful ways: number of tickets opened, number of defects reported, raw performance metrics, uptime, availability, quantitative user feedback like NPS, and engagement metrics like product usage or feature usage.

But there are caveats: you can't turn these metrics into goals. You need to lead toward the improvement of the systems responsible for those metrics, with the hypothesis of improving them and ultimately the end goal of providing a high quality product of value to your customers. The metrics are valuable information about how the system operates, not the end goal.

There's another problem: you're absolutely right about quality being fundamentally qualitative. It is, after all, the root of the word "qualitative." That's significant. Quality is the humanity of the engineering game: while you can measure a lot of things toward it, ultimately, as W. Edwards Deming said, "The most important things cannot be measured." It sucks, but it is closer to reality than defining any metric or set of metrics. Instead, it requires leadership and a comfort with doubt and complexity. Here's my answer to a Quora question expressing discomfort with this paradox: http://qr.ae/f5iUg

So it's a bit of a Bermuda triangle. First, there are significant metrics you can measure, and you need to decide which ones mean quality to you and your customers—I promise they exist. Second, you can't aim directly for your metrics, since any large enough organization is highly complex and over-optimizing for a single metric or set of metrics can cause significant unintended side-effects. Third, the most important things critical to quality actually can't be measured; or measuring them would cause more harm than good. This makes for a challenge, and it's why this is such a difficult problem to solve in an organization.

What I'd LOVE to see moving forward is instead a profound comfort with complexity, and an understanding of why numeric metrics aren't always the most vital ingredient to success.

Reply on news.ycombinator.com