I hope you're not in charge of hiring QA. Bugs are often found by people who AREN'T thinking like the developers whichstart wearing blinders on how they're stuff should work and stop trying stupid things.
I hope you're not in charge of hiring QA. Bugs are often found by people who AREN'T thinking like the developers whichstart wearing blinders on how they're stuff should work and stop trying stupid things.
I think this sort of excuses the initial blindness to Spectre style attacks in the first place, but once the basic pattern was clear it doesn't excuse the subsequent lack of discovering any of the subsequent issues.
It is as if someone found a bug by examining the assembly language of your process which was caused by unsafe inlined `strcpy` calls (though they could not see the source, so they had no idea strcpy was the problem), and then over a the subsequent 6 years other people slowly found more strcpy issues in your application using brute-force black-box engineering and meanwhile you never just grepped your (closed) source for strcpy uses and audited them or used many of the compiler or runtime mitigations against this issue.
This seems unnecessarily harsh. The whole post would be improved by removing that sentence IMO.
In fact, chip-makers famously invest an insane amount of money into "QA" (aka "validation") and many features or lack thereof are often put down to the cost of QA rather than the cost of development.
Sure, umbrella, but different teams. What I'm trying to get across is that it's a good thing that the developers are not wearing the QA hat, so that the QA people are thinking in ways that aren't exactly parallel to the developers. You get close to your work, you sometimes forget that you're not look at the larger picture.
Think of it like if I was building a big secure wall in front of my house, but I neglect that it's easy to just walk to the opposite side to get in. Oh, whoops, I was too focused on the solution in front of me.
Just fundamentals is all.
I guess you could say this was "under the umbrella of the chip-maker", though we had little say in it aside from pressing them for progress as our final product's shipdates came and left. When we finally got the first samples, power consumption was, I think, an order of magnitude higher than expected. Our lead engineer struggled to get it down without going to a smaller process that we could barely afford (and given the delays already, could probably not have afforded to wait for). We thus thought we had working logic, but our case designs were scuttled. After enlarging the cases to accommodate extra cooling, our base unit was more than ten times taller, and our next size up, while the same height, was three times longer. Highest units had a water cooling system[1].
Our QA was able to find other sorts of flaws, like misprinted unpopulated circuit boards, software faults on the host, or when we received shoddy interlink cables that either melted under test[2] or other cables that scrambled communications[3].
All of which is to say, that many other pressing issues can interfere with doing what you feel you ought to be doing. At least at our scale. I can't speak for the likes of big guys like Intel or AMD, but it's possible that unfound faults or known unpatched flaws can ship because resources were committed elsewhere or fabrication leadtimes preclude waiting. This is not to say that shipping a security flaw is okay, but rather that sometimes you think you've done you're due diligence, or sometimes your choices seem to be "Ship, ship late, or never ship.". The answer you pick can be existential, so you hope you've picked the least bad option.
[1] Misbegotten, because "beauty of the promo images".
[2] Conductors much thinner than spec, not initially observed because both ends were fitted with moulded plugs.
[3] Longer than spec, initially recieved with enthusiasm by assembly staff, before an engineer investigating a difficulty saw them and exclaimed, "No-no-no! That's longer than I am tall! Stray capacitance alone will kill the communication.". (He uncharacteristic'ly exagerated here. While they were three times longer than expected, this was at most .40 times his height.)
Why the personal attack? The logic is sound. Isn't QA typically part of the same organization?
Generalizing: QA/Test and dev people just think differently.
I served as QA manager for a while. I was fortunate to have worked with some really, really good QA/Test people. They're more rare than good devs.
Some individuals can do well in both worlds.
I've had great devs who were pretty good at test. It seems to me like these devs came from outside of software and CS. Like from aerospace or ballet or history.
I've mentored QA/Test people so they could better automate and manage their work. Then they could pick up tasks like CI/CD, testing scripts, scrub data, etc.
But, in my experience, devs are bad at testing, and just terrible at QA.
Getting enough QA/Test support on a team in the 90s was a tough sell, even though everyone gave lip service to quality.
It's been a long time since I've worked with an actual Test person, dedicated to that role. And that was just 1 person vs 8 devs. So ridiculous.
These days, any kind of "test" is done by "business analysts", whatever that means. And I can't recall the last QA person I've worked with (since my stint as QA Manager in the '90s).
FWIW, I wholly agree with Dan Luu's observations about today's QA/Test standard practice in software vs hardware.