Notes on Software Quality
anthonyhobday.com
anthonyhobday.com
If you can get 100 experts to agree on something then you've cracked a much harder problem than software quality.
The 1 in 10 who didn’t recommend $PASTECORP made up the 9 in 10 who recommended $NUTRAPASTE
There is over 30 different variants of Colgate Toothpaste, do dentist believe ALL the Colgate ones are better than the alternatives?
Most people with bad vision want higher contrast for text, while most people with dyslexia want text with lower contrast.
So, if 100 experts can’t find a problem, the thing is probably perfect. Unfortunately, this will never happen, because perfection is impossible.
(in the case of curl, any security issues inside the library should be mitigated by its environment)
I didn't see this addressed in the article, and it could be more important than some of the main points.
>Quality is the absence of problems
A low quality code base can be problem free if surrounding circumstances are forgiving enough. Conversely, a high quality codebase can have a lot of problems in difficult circumstances.
I haven't thought about it long enough to have a definition of quality that I'm really happy with, but I think a "resilience to hardships" would be a better definition of quality. Hardships can come in many forms, and often you're prepared for some of them but not all. Occasionally you'll be prepared for hardships that never occur. There is something to be said for being resilient against the correct kinds of hardships, which is why I'm not entirely pleased with my definition either.
But absence of problems is not it. That might be entirely circumstantial and is therefore orthogonal to quality.
There isn't some intrinsic value to software, it's gotta be used by somebody
Yes, such software exists.
Software quality is, roughly speaking, intrinsic to a software product. Its usefulness is not intrinsic, it can depend on any number of external factors. Quality is not the same as usefulness.
If I were to write 2 simple functions that reverse an array, and one successfully reverses an array and the other just randomizes it - despite never using this function, we could all agree that function 1 is of higher quality than function 2.
This is an extreme example but the same can be done for, lets say, speed or error handling.
If that code never goes into production with real customer use-cases, everything we say about its quality (on any metric) is 100% theoretical! We can try to replicate that environment, and we should, but until it's actually used, we can't truly know.
Does this refer only to program behaviour? I figure readability should count toward quality, but it doesn't directly affect program behaviour.
i disagree, think about what defines a problem. Not being maintenable, readable, performant etc could be problems or may not be depending on the software requirments.
> Occasionally you'll be prepared for hardships that never occur.
this over-engineering and just as bad as failing to meet a requirement, you're wasting resources that could be spent on something else. In fact, meeting the requirements and only the requirements is requirement #1 ;)
Properly speaking, that would be a characteristic of the entire production process, including the people, rather than a property of the code itself. (At least for now. Stay tuned with AI for further updates.) Still, you'll see it in the code.
Software very rarely exist in a vacuum
I once saw an organization that migrated from Weblogic to Kubernetes, keeping the rule to have a least two instances of each app but forgetting that that their Weblogic had distributed transactions and without implementing something similar in Kubernetes.
They happily serve users, helped by low traffic and low expectations. Occasional data corruption and race conditions give work to the support maintenance teams.
Quality is a set of characteristics that are ultimately decided by the user, and not every user will agree on what those characteristics should be. This is why user testing of all sorts is very valuable.
You have to go back to Walter Shewhart's work and writings while at Bell Labs. In 1931's 'Economic Control of Quality of a Manufactured Product' he draws a line between 'a thing' and the human experience of it, drawing heavily from the philosophy of C.I. Lewis:
"There are two common aspects of quality: one of them has to do with the consideration of the quality of a thing as an objective reality independent of the existence of man. The other has to do with what we think, feel or sense as a result of the objective reality. In other words, there is a subjective side of quality."
He draws a line between objective quality (the measurable, physical things) and subjective quality (the human relationship to those properties).
Later in 1939's 'Statistical Method from the Viewpoint of Quality Control' he gives a good example:
"Let us take a very simple example. Suppose we consider the quality of a glass of water. We may list some of its characteristics as follows: temperature, volume, chemical purity, taste, clarity, and color. If we look at any one of these characteristics, say temperature, we see that it is impossible to specify the temperature of a glass of water to the last degree of accuracy."
I would argue this is the standard to go for when looking for a definition. Ask yourself what makes a quality glass of water and you'll immediately realize you need to ask 'according to who?'.
The distinction between "measurable objective" vs. "perceptual subjective" is fundamental to understanding quality.
To add to the above; From https://en.wikipedia.org/wiki/Software_quality -
The transcendental perspective deals with the metaphysical aspect of quality. In this view of quality, it is "something toward which we strive as an ideal, but may never implement completely". It can hardly be defined, but is similar to what a federal judge once commented about obscenity: "I know it when I see it".
The user perspective is concerned with the appropriateness of the product for a given context of use. Whereas the transcendental view is ethereal, the user view is more concrete, grounded in the product characteristics that meet user's needs.
The manufacturing perspective represents quality as conformance to requirements. This aspect of quality is stressed by standards such as ISO 9001, which defines quality as "the degree to which a set of inherent characteristics fulfills requirements".
The product perspective implies that quality can be appreciated by measuring the inherent characteristics of the product.
The final perspective of quality is value-based. This perspective recognizes that the different perspectives of quality may have different importance, or value, to various stakeholders.
I know where you're going with this, but here's a twist:
A CEO who cares about interface _design_ is path to micromanaging and pain. A CEO should care about interface _designers_, who are (hopefully) the people trained on how do it well.
Even better: CEOs should care about developers with UI/UX skills, because too often CEOs adopt designers like a pet and keep them busy 24/7 asking for mockups.
That's how Apple blew up into a trillion dollar company.
Plenty of other CEOs have thought the secret to Apple's success was micromanaging like Steve Jobs and been proved very wrong.
The best CEOs hire people smarter than them (in their respective disciplines) rather than assuming they always know best.
I've even seen a version where the designer is the CEO's daughter, which is the worst possible version of this pattern.
I'm interested in quality, but I didn't find these notes enlightening, and couldn't even finish the article.
Personally I find quality to have a fundamental impact on everything every human does. It affects mental state, motivation, affects ability, necessity, and time to do things, creates or reduces costs, availability of resources, clarifies or complicates, makes life easier or harder, etc. It can save or destroy a business, make someone's life feel easy as pie or insanely frustrating. But it's not always easy to do right; you need a system to apply quality intelligently or you risk your efforts being wasted (https://global.toyota/en/company/vision-and-philosophy/produ...).
^ = from your linked Toyota Production System philosophy
Thanks for that link! I find those two early sentences to be an insightful and relatively complete loop of process.
When considering using automation or ~A.I. we can easily ask: which part of this loop is our addition improving (or messing up)? Where is the balance that works?
As you point out, answering these what-works/quality questions are not solved problems and expertise is needful, but careful consideration does not seem to be a popular mode in our age of fountaining funny money and magic genies.
I grew up in the '60s where the science fiction/future was always: march of progress and you'll have so much time and choices! Now I am in the future and the reality is close to: who has the time?
More insidiously/invisibly: you have plenty of time for endless momentary thrills, but no one has time to make things better.
Once upon a time there were customer complaint departments and the Production System would get fixed. Then it became suggestion boxes and returns. Then pleading for a Return Merchandise Authorization. Then it's your call is valuable to us recordings before click call hangups. Oops, unhappy customer?--give 'em a coupon to keep up the addiction, it's cheaper than Quality Control.
The latest devolution seems like fire the workers but use AI to mesmerize customers, or just mind control ~investors and ~regulators, since who needs cover-our-costs-paying-customers anyway?
Will the pendulum swing back?
For me it means care and attention were paid while developing, the rough edges have been smoothed off for want of a better phrase. This doesn't mean using the latest and greatest framework or library, usually quality will come from a deep understanding of the basics and concepts like design patterns .
You can spot quality code in the same way to can tell a fake Rolex from a real one but the quality of the movement.
That's a "yes and no" thing. Handmade quality, yes, but some companies get pretty good at finding a "sweet spot," between better-than-average quality, and "Rolls Royce" quality.
Source: I worked for a company that was pretty much renowned for Quality. We made stuff that is pretty near the top shelf, but still a rung or two under the top.
It's not easy.
Also, customers are willing to pay for garbage. As long as that continues, garbage producers will drive quality producers out of business.
This is why a better definition is "quality is value to some person who matters." This definition instantly places you at the crux of the matter, which is not about a state of the world or of your product in the world, but rather about WHO matters and how they FEEL about your product.
> The quality of a system is the degree to which the system satisfies the stated and implied needs of its various stakeholders, and thus provides value. Those stakeholders' needs (functionality, performance, security, maintainability, etc.) are precisely what is represented in the quality model...
Here is a decent summary: https://iso25000.com/en/iso-25000-standards/iso-25010
Not exactly software quality, but adjacent, is the capability catalog of dora: https://dora.dev/capabilities/
Dora is about 'software delivery and operations performance' and has a vast body of empirical research underpinning it.
So all that remains for our so called “engineering” discipline, is an answer that says something that doesn't break a lot.
Sure you didn't miss one? You can't have an exhaustive list because any of those can be just as true as false depending on the situation.
Instead of picking the ones I disagree with most, I'll just say that low quality is miscommunication. The bugs are a snapshot of the organization.
There are multiple facets to hang concern on that the other stakeholders don't know about or ignore. Your ability to discuss them, plan, and execute is the bottleneck. Everyone has to be on the same page.
This cannot be the sole responsibility of the devs or small isolated teams. Scale is necessary for quality to emerge.
Most typical quality attributes can be subsumed under this schema.
If the software costs too much to change because the architecture is poor, that reduces the quality.
If the interface is poor or it is slow unnecessarily, it costs you more time to use it, and that reduces its value to you and hence its quality.
If the software is insecure, you may actually lose privacy or even money using it.
If the software is not robust, you run the risk of losing data and time, etc. when using it
Some of these are difficult to quantify, but are often the difference between success or failure, in the market.
I constantly encounter “dead” software. Software that is correct, performant, awesome (in some cases), but something that I don’t “want” to use. A “necessary evil.”
That kind of statement doesn’t fly well, in a community of “Inspector 34”s, but it applies to those we like to call “customers.”
And "simplicity" comes to mind. Also not mentioned.
Quality is an opinionated measure of a design, which involves tradeoffs based on your goals, values, etc. Valuing security very high, even at the expense of every other aspect of quality if necessary, is completely understandable. But that’s a value that isn’t universally held.
If your point is that security should at least be considered when measuring quality these days (like, it’s a top 10 ‘ility’ at least): fair enough.
The what? Since when has "the industry" been able to define best-designed, much less agree on it?
Imo this point should be changed to reviewability. Beauty is in the eye of the beholder
“How likely are you to work in this codebase again?”
0 = Least likely 10 = Most likely
baseless speculation follows!!!
I think large orgs can definitely achieve high quality. but only by spinning up small, totally autonomous teams working on every layer of whatever stack their product is on, one team per product (maybe two if there's some really obvious line in the sand between two different things that talk to each other, but be careful! and make sure both teams are in the same timezone!).
As soon as you start trying to do those things that seem really sensible when you have a bunch of separate autonomous teams - like "hey you're both working on similar features, you should share the implementation", and "oops all our products look different, we should come up with a unified component library", and "we need automated tests - everyone should use this specific tool that we paid for" you run into the big org problem.
My gut feel is that the best way to get some level of coherency without running into these problems is to share knowledge, best practices, examples etc. But never dictate anything that actually gets in the way of any of your teams owning their own shit. Don't make teams use some internal/external library for functionality x, don't enforce processes, don't have a separate design team dictating css styling to teams, don't enforce org wide CI policies, don't have a separate DevOps team handling releases - just hire competent people and let them do their thing. If you do want to try to build something so that all the teams solve the same problem in the same way, you need to get them to use it by making it so good they want to, not by telling them they have to.
You might be able to enforce some baseline level of mediocrity by doing those things, but the only way to achieve excellence is to get out of the way and stop trying to "help".