More importantly, it is almost impossible for engineers to be as well incentivized to spend extra time exploring edge cases in something they already believe to work than to ship a feature on time.
Like everything else though, its contextual. Complexity of domain, surface area and age of product, depth of experience on team and consequences of failure are all so variable that there cannot be only one answer.
I have done it both ways for many years. I have worked on teams where QA is a frustrating nuisance, and teams where they were critical to success. I have worked on teams that did pretty good without them, and probably those were the highest throughput, most productive teams because the engineers were forced to own all the consequences - every bug they shipped was a production issue they were immediately forced to track down and resolve.
But those were very small teams, and eventually I was the only founding engineer left on the team and far too many mistakes by other people made it to my desk because I was the only person who could find them in review or track them down quickly in production. That was when I started hiring QA people.
Personal liability and professional insurance works for all the actual “professions” in the US, to some extent, right?
It might be time to start the considerations for professional licensing for platform scale or commercially published software.
Medical devices and such are the only places I’d expect to see the need for certified products. By extension, in the new era, we really ought only expect certified software where we expect a duty to care from the software system (or any other assigned duty)
Usually there either arent any in which case bugs get missed or there are 5 very cheap ones running mindless scripts who are standing in for the devs' inability or unwillingness to write decent automated tests but dont catch the really deep level thorny stuff.
Hard disagree. QAs' main tasks are acceptance testing and V&V. This means their work is to act as independent parties whose role is to verify that the results are indeed being delivered. Their importance is up there next to Product Owners.
The problem is that some QA roles are relegated to running the same old test scripts and do some bullshit manual exploratory tests that catch and assert nothing. It's the ultimate bullshit job as it's performative. This is clear in the way that LLM agents are seen as threatening the very existence of the role.
When I got started in software, QA was already in a heavy decline. A mentor who had been a QA manager at Apple told me that being a QA person (in the industry) was once seen as a high-trust position. He was sad at what it became.
Their approach to QA was to have this be a optional thing the service people could do.
It worked surprisingly well, with the caveat that they never created regression tests.
The employer eventually switched someone from that team to establish these regression tests full time, but they had no programming experience and by the time I left no real progress was done in around 6 month. No idea what came of that, and a few years later they fired a lot of the team
But there is also bad QA: The most worthless QA I was forced to work with, was an external company, where I, as developer, had to write the test sheet and they just tested that. Obviously they could not find bugs as I tested everything on the sheet.
My most impressive QA experience where when I helped out a famous Japanese gaming company. They tested things like press multiple buttons in the same frame and see my code crash.
This was my sole experience at the one place I worked with an internal QA team. They absolutely could never find bugs that devs missed, often mis-marked ones that didn't exist, and failed to find obvious edge cases that did exist.
Multiple devs fired because the CEO believed the QA over the engineering team; if they marked a bug as present, it was the engineer's fault for writing it. If they didn't catch a bug that made it to prod, it was the engineer's fault for not including it in the test plan. They represented nothing but red tape and provided no value.
Good QA sounds great! I'd love to know what that's like someday! It'd be great to have someone testing my code and finding breakages I missed! I'm only slightly (incredibly) bitter about my bad experience with its implementation.
This is an area where I expect AI to create a bimodal future. The smaller group of high quality QA people will now be able to offload the activity to agents instead of the QA drones. They'll still be worth their weight in gold, whereas the drones will be redundant.
The latter is much more high touch, but they're often worth their weight in gold. The former is kinda pointless.
Moreover, the best QAs would almost always try to be not QA - to shift into a better respected and better paid field.
I wish it werent so (hence my username) but there is a definite class divide between devs and QA and it shows up not just in terms of the pay packets but also who gets the boot in down times and who gets listened to. This definitely affects the quality of people.
I think it's overdue an overhaul much like the sysadmin->devops transition.
This might have been an Apple/MS thing, but we always had very technical QA people on the dev tools team. For example, the QA lead for the C++ compiler had written their own compiler from scratch and was an amazing contributor.
This was all unfortunate, and I agree in principle with having a separate test org, but in Windows the culture unfortunately seemed to be built around testers as second-class software developers.
As I said above, everyone has their own experiences but the QA folks I worked with at MS were fantastic.
Not sure if you're aware but Dave Plumber now has a really good YT channel [0] where he talks about MS back in those days. It's a fun walk down memory lane.
But yea, so many companies cheap their QA and then wonders why their QA sucks.
> Moreover, the best QAs would almost always try to be not QA - to shift into a better respected and better paid field.
That sort of seems circular. If they're not respected or paid well, of course most of the talented people would not want to remain in QA, and eventually you'd just have mediocre QA. That doesn't really give you any insight into whether high quality QA would be useful though.
(edit: I see now that's basically the point you're trying to make, so I guess we're in agreement)
> They don’t always make me happy (in the moment) with their bugs or with how they decide to break the software, but in the end it makes a better, more resilient product.
I think the core of the issue is the weasel word "Good QA" instead of just "QA". You're underlining the importance of having someone in a team who has good product understanding and has good ownership skills. How many QAs fit that description? Not many, unfortunately. Some FANGs outright eliminated the role and replaced it with a mix of product owners and team ownership, and some QAs are just doing their 9-to-5 job going through their manual test scripts to verify it something meets a definition of done. What happens when you can have a LLM agent doing the same instantly as a step in a random CICD pipeline?
This is a big asset in the current paradigm where a LLM agent can effectively implement behavior only when given a robust test suite to iterate against.
I don’t understand the reasoning here why QA shouldn’t be engineering.
Who watches the watcher, right?
That aside, the core idea is the same as the principles of independent audit, peer review, or even simply just specialization.
Red team / Blue team?
Even the military have police, right?
edit: ultimately, it comes down to the importance of independent audit, the builders and the breaker/fixers are very different groups in engineering.
Nor, in the case of QA, should the audit team be engineers trained to act and think like the ones who wrote the software. A fresh perspective is useful.
But in the long run, supervisory independence is the real deal. I know of a QA manager who shut down an entire factory's output until a major safety issue (that had been kicked down the road several times) was addressed. It took chutzpah, and serious power, to do that. The Dir. of Engrg. would NEVER have allowed it.
It's not a protected term in the US so it's jarring to those of us living where it is.
This is funny.
What I have seen before with QA is a queue like system. Hand to QA. get back in 3 days. Not the QA persons fault. It is the setup if the org that is wrong.
Imagine if you had a code review role and code review team and you had to wait to get your code OKed. We wouldn't put up with it.
QA teams seperate from Eng worked when the software was burned to CD rather than uploaded to s3.