Hacker News Reader

Comment by s-luv-j | Hacker News Reader

>"You can not inspect quality into a product." ~Deming

Yep. This applies double to software. Putting software under load and trending health measures to look for two standard deviations variance is very similar to how Deming instrumented production lines. The neat thing that software enables for both automobile manufacturing and software products themselves is that those exact same health measures can be trended and analyzed after they get into the hands of consumers.

If your quality apparatus is purely inspection based, you will live your life one escalation to the next.

View on HN

Deming is talking about repeatable manufacturing processes. The point of Statistical Process Control is that you sample metrics that tell you if the process will produce products reliably within tolerances.

Software is completely different, it is hardly ever a repeatable process to create software. Making the same software over and over is nonsensical, but making a billion identical contact lenses is exactly what you want.

There are forms of software "inspection," like pair programming, that amount to what Deming says you should not do in manufacturing: Inspect everything. But pair programming is pretty rare. Other practices like code review have been called into question. So it may still be true that you can't inspect your way to quality in software, but not for the same reasons that it is a bad idea in many cases in manufacturing.

> Software is completely different, it is hardly ever a repeatable process to create software. Making the same software over and over is nonsensical, but making a billion identical contact lenses is exactly what you want.

I've heard it said like this: Creating a software is like developing a recipe for a cake. Baking thousands of cakes (a "manufacturing process") is what you do once you have the recipe. The two activities are very different.

Kind of. We have to clarify what we mean by "creating a software".

This seems to be rarely done in most discussions on processes for software development. The lack of information regarding this means that everyone gets to be right and angry at everyone else.

Are you making novel systems? Then it's like developing a recipe for a cake. Are you making bog standard CRUD web apps? Then you're baking thousands of cakes.

A lot of software development sits somewhere in between these two extremes. It has repeatable, well-established portions that can be executed almost mechanically, and portions that require you to sit down and collaborate or really think about what you're doing. How these balance out on a project determines what approaches are appropriate to even consider.

Then there's software maintenance (an unfortunately neglected aspect of software in many parts of this industry). Here the same consideration applies. Are you writing new software that's really just generating new kinds of custom reports? That's rather mechanical, you may even be able to automate yourself out of a job. Or are you adding truly novel features, extending your document editor to support real-time networked collaboration?

Software runs the full gamut so no statement about what software is will ever be correct. We can only discuss the utility or applicability of processes once we specify which kinds of software activities we're involved in.

Certainly some software looks routine and feels repetitious while you are doing it. But very little if any software is amenable to determining its manufacturing tolerances. That phrase is jarring because the idea of manufacturing tolerances for software does not make sense. But manufacturing tolerances are fundamental to statistical process control, which then builds on statistically valid sampling to determine if part of a process is failing. (Or it can determine that a quality issue is something other than a process failure.) But it depends on setting and using manufacturing tolerances wisely.

You are 100% right that measuring the process of creating software is a terrible application of statistical process control. See Jira.

One way to look at how Deming/Shewart's work applies to software is to consider what happens when you apply their statistical analysis techniques to running software.

I've spent much of the past few years trying to work out if this means I should stop relying on pull requests to catch problems, and put something in place ahead of writing code instead, but so far I've not figured out an alternative that works. If anyone has any ideas, please share them.

One of the reasons why pushing change straight to production with A/B testing works so well is that you can expose your new change to statistically significant real user load and, assuming your have decent instrumentation, find the problems. This does mean that a subset of your users get exposed to those problems.

A more user friendly way would be to have a scaled down replica of production setup with simulators that mimic user behavior. Assuming the development team actually believes in and tends this environment, they will find the lion's share of bugs that commonly slip through peer review.

Pair-drafting pseudocode with a colleague into a Dropbox Paper document is a great way to evaluate tradeoffs.

Reply on news.ycombinator.com