This is probably not universal, though - what sides of the fence of org design do people fall into, here?
Eventually QA ends up being owned by either the Engineering team or a standalone QA team. I've seen both work, and both not work. Generally if it's not working for Engineering, it means they're not automating enough or the tests are flaky and they're just ignoring it. If the QA team approach is not working, they're probably overworked... Likely because the engineering org has deeper quality issues they need to fix, but they're trying to make up for it by throwing bodies at it in the QA org.
* We have a no-code test automation product that is used by both QA and Engineers: https://reflect.run
In my view, the best approach is rigorous automation and manual exploratory testing which will find the kind of bugs automation misses and inspire new test cases.
I think automation is often a waste of time at the beginning of a project too. Bugs don't cost much (if anything). Building functionality that gets tossed away is routine. Tests don't provide value - getting the code in front of customers and seeing if it's what they really want provides value.
The problem is that once people get used to not having automated tests is can be difficult to build up the habit.
Unit tests are also often hard (if not impossible) to retrofit, although IMO that's a bigger problem with unit tests than it is doing testing at a later stage in the project.
I am not sure _why_ this is but I think it has something to do with the "Not my problem" mentality where engineers say "QA will figure out if this works or not" instead of "I'm responsible for figuring out of this works".
It hasn't become a red flag for me _yet_ but it's definitely _a_ flag.
Maybe it's because we were embedded in the dev team.
Sorry you didn't experience the same thing.
That said, an understaffed QA team can be really helpful. Not enough people means you can't ship garbage and expect QA to catch it; they'll catch many things, but not everything. Not enough people also encourages efficiency in testing --- test the most important stuff well, test the other things opportunistically.
A QA team also can really help turn customer problem reports into actionable bug reports.
When I've been on teams without formal QA teams, the team was focusing way more on having good automated test practices in place.
This also applies from the SDET/SDE separation, which I see someone else in an adjacent thread mention. I've found a lot of value from taking ownership of both the code and the automated browser testing code. I found when those roles are separate, the browser tests end up breaking too often due to developers not being aware of how coupled they are to the current implementation. I also found when a dedicated team is writing browser tests, it becomes easy to go overboard with how much you test.
I don’t think this is an anomaly. Every engineering team I’ve been on since has given this responsibility to the developers writing the code, and every one has been stronger for it.
I try to submit feedback and half the time the feedback tool itself fails to work, the other half of the time my bug is closed or ignored so that the team can 'focus on issues with a broad customer impact'.
Is there any evidence that Microsoft's product quality has improved recently? The only way I can imagine justifying that point of view is if you divide by the cost to get that quality, which is a useless metric to me as a customer.
Maybe I just experience bugs that no one cares about, but I no longer recognize Microsoft Office or Windows as solid software given the massive increase in the volume of bugs I experience.
My MS friends have the feeling that it was a painful transition, yet ultimately a good thing, and that quality has gone up.
My experience on the ground, so to speak, is the exact opposite. The May(!) update still has showstopping bugs with a not-small portion of my clients. Last year, one was bitten by the wipe-the-user-folder upgrade bug. The trust in running windows updates is at an all-time low amongst the general users, and I can't blame them.
I trust my friends. I also trust my own experiences. I wonder where the disconnect is?
I'm sure that's what the execs say at the all-hands meetings but ask the former MS SDETs who still use MS products and you'd hear a much different opinion. ;)
My standard quip when hitting yet another bug in a MS product, which is an all too frequent occurrence nowadays, is "Gosh, maybe MS should consider hiring some SDETs."
If you have good engineer-written and maintained tests, then you don't need QA.
The problem comes from relying on developers to do manual tests. It works as long as the product is simple and newcomers can learn the entire product. But, developers doing manual testing won't scale once the product becomes too complicated to remember all the corner cases and gotchas. Even worse, newcomers just won't understand the product as well as the people who originally wrote it.
So, if you have a complicated product, and a lot of debt in developer-written tests, then you need QA.
With regard to the web, it's very easy to write automated tests against server-generated HTML or an HTTP API. In contrast, other platforms are all over the place in how easy / hard it is for a developer to write an automated test.
Funny, I find the opposite. Code for the web is hard to write reliable tests for. Typically I want to test that my component actually exists and is visible and does the right thing in a browser, so I need to produce the whole page instead of just the unit under test. This means that bugs - or just changes - in other parts of the page can make my test fail.
I'm not a front end expert and my colleagues rarely have been either, so maybe I'm missing something about best practices here, but I've definitely had the opposite experience to what you describe.
1) They are expensive. You can hire 3 or 4 QA people for every dev a lot of times. Why waste that money on devs?
2) They aren't that great at. They know how the program works, so most struggle to do things outside of the "happy path". I had a QA (actually my PM) paste in an entire book into a text field and break the site that way. Devs don't think of stuff like that. Another uses the browser's back button constantly, which none of the devs seem to do.
That said, it doesn't excuse crappy code. If a code change/addition fails QA, that's on the Dev. However, if that failure hits Prod, that's on QA...
Developers aren't paid to test, the majority of their time is spent making new code because that's what management/customers want. Having one person have to wear 2+ hats doesn't work when they can never take one of the hats off.
If they were legitimately given the amount of time to test as they get to code, your project could take twice as long, but it would be more thoroughly tested. And, likely, it wouldn't take twice as long because they'd actually have to think about testability and design earlier. And by "test" I do not mean just unit tests. Designing and implementing integration tests, assembling regression test suites, etc.
Devs focused on just unit tests will never catch the errors that a dedicated V&V team can find.
Attitudes like this will get you...not great QA. In my experience a sufficiently strong QA person can both cost and be worth more than adding another dev to the team.
In my experience, QA teams are usually starved of engineering resources, so a lot of things that can be automated are done manually or the automation is janky and constantly breaks. Once these issues are brought to the forefront, it becomes a priority to fix so that your expensive engineers aren't wasting time chasing bugs or fixing broken automation.
I think we have a good code-quality/speed-to-release ratio with well known tech debt that is actively being cleaned out, that is also tested after refactored out.
Having QA Tester be part of the team makes it really easy to adjust features, adjust the release if an update is needed and it is not passing quite yet quality wise. It also makes it really frictionless specifying what to expect from a ticket/feature and how to baseline test it, the tester also performs some tests of his/her own, the QA tester runs unit, automated UI and performs manual regression tests.
I've been part of teams that have QA testers outsourced as a dedicated independent team and it definitively impacted code quality and the release turnaround, mostly because of delayed communication and mis-alignment on what to test or the expectations of the test.
The process we used was the three amigos process, where the ticket dev, BA and a QA would have a 20-30 minute sit down and draw up a mind map of test cases linked to the acceptance criteria.
Once the mind map was created, anyone technical could pick up the actual writing of the tests/exploratory testing, whether that's the original dev, a QA, or any other dev. Because devs were all testing each other's tickets instead of just hoofing them over the fence to QA I found that I got a much better understanding of the different areas of the projects.
Only downside was it required a lot of meetings to really flesh out the tickets and make the acceptance criteria as detailed as possible.