Want Better Quality? Fire Your QA Team

My wife has a freelance QA consultancy, and while I understand that there is often the sense that QA is a 'nag', I can honestly say that whenever she is involved in my work, the quality of my output is much better, and takes less time to get it there.

A good QA analyst / team does much more than point at a problem and say 'there it is'. A good QA analyst can determine why you made an error, and can anticipate other errors you might have made in the same way. A good QA analyst will learn your formulae, learn your equations, understand your workflow and try to find root cause, even if they're not developers.

Having a good QA person with a trained eye for knowing what I'm likely to have overlooked is also FAR more cost effective than your high-paid developers doing the same task. If they know that 'heads will roll' when they get errors, I'm also betting that makes your releases even slower.

Off-topic completely, but I generally prefer non-developer QA analysts, as they generally prove themselves to 'think too much like me' to be effective.

> This app can't go down, and it can't be wrong.

Shops who set the bar this high and mean it are exceptionally rare (I look for them when I'm on the market). Where I am now, an hour of downtime costs us my salary for weeks, and still we cut corners that don't always sit right with me.

But having generalist engineers create functional and integration test plans? Not only do we tend to cost more, but we have less experience at it than QA specialists. And a dev shouldn't test their own work, because they won't notice any unwarranted assumptions they made in design and coding.

ridiculous.

a large fraction of errors that turn up in QA are conceptual errors made by the development team; like the blind spot in the eye, you don't know what you don't know, and there are certain problems that you'll never find yourself.

Phil Crosby would say that management is responsible for quality; management writes the specifications and has the responsibility to hire and fire developers and QA people. The buck stops with management.

Efficient quality requires practices to be work correctly across the organization, but it starts in the specification process. If you don't know where you're going, you're sure as hell never going to get there.

If a developer's reaction to a client-facing defect is "QA didn't catch it", you have cultural problems that firing a QA team probably won't fix. If you don't have a culture of personal responsibility, a finger can always be pointed elsewhere.

Developers should absolutely write and use unit tests. Where QA really shines in the larger scale: validating actual business requirements.

I 100% agree with this article. Quality is everyone's job, not just the QA teams' and I believe that, in most cases, developers can and should be responsible for developing, testing, and releasing their code. When developers can't "throw code over the wall to QA" because there is no QA, they will do a better job testing it themselves.

I don't buy the popular opinion that developers can't test their own code like there is some sort of magic that goes on in the mind of QA engineers that causes them to be able to find bugs that developers couldn't find themselves. In fact, I think the opposite is true - developers are better suited to test the features they are writing because they know the code and know the risk areas, where things need to be regression tested, where more attention should be placed, etc.

I do think that it can be helpful for another set of eyes to look at a developed feature and this is where pair programming can be useful. It does not need to be QA doing this.

One caveat is that this approach does not work for all teams. This works for small, agile teams where roles and responsibilities are less defined. If you fire your QA team but your developers can't test their own code because they are set in their ways of writing crap because they expect QA to find the bugs, you may end up firing your dev team too....

QA and dev are two different skills. It's like saying that we save on design by having the devs do it, or having your quarterback also play noseguard. Sure it works in high school, but not when you're a pro.

I'd much rather hire someone who knows testing and QA well and let my devs focus on writing bug-free product code.

By the time anything gets to the QA person at my dayjob, the devs have usually tried every damn thing that they could think of to break their own code.

The usual reaction to the bugs she finds is "what made you think of trying that?"

I work in QA in a high-reliability industry. Correct operation of some our products saves millions and saves lives. We need super high quality. How we do QA is pretty traditional. In particular, we have functional test teams. The perspective we take is that the testers take the specs and develop tests to exercise the functionality, and theoretically, we (QA) cover any spots the developers missed, as well as providing assurance that the developers aren't talking out of the wrong hole when they say they're done. Seems to work most of the time except when mutual brain-farts develop.

I don't know that I would go so far as to recommend firing myself to increase quality. I would have to visit with many teams and really gain experience in the way they work. I do think that we could take a step forward by having a standard across the company to have fully automated test suites running continuous builds and tests. That, however, runs facefirst into the problem of Legacy Stuff and ROI. Automation just plain takes time when you are interfacing against external programs. It's not always worth it in the short/medium term.

There's no silver bullet to the software quality problem. But me, I think that continuous integration provides a copper bullet.

This works as long as your developers are talented in QA and are motivated to do it. That sounds like it requires a lucky constellation of events.

Don't be so quick to brush this off. The argument isn't so different from saying that we don't want to distinguish "front-end" and "back-end" programmers. It's probably a special circumstance that you need a team that is entirely dedicated to verification.

Feynman talks about this arrangement in his essay on the shuttle program (http://www.fotuva.org/feynman/challenger-appendix.html). In a nutshell, the core software team is responsible for checking the code. Then there is a totally removed group that is adversarial, i.e., it is measured on its ability to trip up the software team. This is the same a "setting consequences" as this author describes.

Now, the shuttle program is truly life critical. I think a good software team can produce quality results internally.

Ah it's a single anecdotal story and comes with no specifics about who it is or in what way things supposedly improved, just some broad calls for firing QA.

> They better get it right, or heads will roll. As British author Samuel Johnson famously put it, "The prospect of being hanged focuses the mind wonderfully."

Gee, sounds like a wonderful place to work. When can I sign up?

My head is more often buried in code than in the product's UI. Our QA team has a better nose for when interaction or UI details are off than I do. In general, they spend more time using the application, and have a better instinct for when a given behaviour is wrong.

Some of the bugs they report would fall under the automatically testable category, but many don't.

I suspect that dogfooding is more effective at companies where the product itself is useful to developers or is a consumer app. e.g. Basecamp, FogBugz, Facebook, Netflix, Gmail, etc.

Ah yes, Forrester.

I hope all of you reading the blog post realise that this brainfart will cost your organisation $5,500 per copy downloaded.

I don't think this would be a good idea. When developing I don't always have the time to be as thorough as a QA team is in testing my own code. It's very helpful to have somebody else with a non-technical background to help spot problems that I am blind to.

Considering I work with one of the most incompetent QA engineers I've ever had the displeasure of encountering, I couldn't agree more. This person wastes my time, and the organization's time, and has not actually increased the quality of software (as far as I can tell). It is a waste of a developer's time to decipher a bug that states "it's broken," without clearly defining "broken," or "it," or how to reproduce the behavior. Often, reproduction involves user error (like forgetting to sign into the vpn). If it weren't for this bozo, our team could get more work done in less time.

I assume that others have had better experiences.

Peopleware has a chapter about "the black team" that was at IBM. The chapter is about team work but they describe how in the process of gelling the QA team delighted in finding programmer bugs. They describe how creation of a reputation and ethos in the black team not only made them a little better than other QA teams but significantly better.

While it's easy to say that QA isn't important many of the QA teams I've worked with weren't respected or valued, and often saw QA as a stepping stone. If you want to increase the quality of the code, don't fire the QA team, motivate them.

What is the metric for quality? Is the article saying that having fewer people looking for bugs resulted in more bugs being found? Or fewer bugs? Or just fewer bugs found?

Most of the other comments already voice my reaction to this post. I will only add that a "heads will roll" attitude towards motivating devs on the part of management would prompt me to switch to some other place very fast. Simply saying "you have to do X as well as Y or heads will roll" just sounds like trying to save money on QA and squeezing more work out of devs - who could be putting that time and effort to much better use improving the product.