Hacker News Reader

I'll espouse the flip side of this:

I've worked with a handful of excellent QA. In my opinion - the best QA is basically a product manager lite. They understand the user, and they act from the perspective of the user when evaluating new features. Not the "plan" for the feature. The actual implementation provided by development.

This means they clarify edge cases, call out spots that are confusing or tedious for a user, and understand & test how features interact. They help take a first draft of a feature to a much higher level of polish than most devs/pms actually think through, and avoid all sorts of long term problems with shipping features that don't play nicely.

I think it's a huge mistake to ask QA to do automation tests - Planning for them? Sure. Implementation? No. That's a dev's job, you should assign someone with that skillset (and pay them accordingly).

QA is there to drive quality up for your users, the value comes from the opinions they make after using what the devs provide (often repeatedly, like a user) - not from automating that process.

Right - the best QA people need only be as technical as your user base. Owning QA environment, doing deploys, automated testing, etc are all the sort of things that can live with a developer.

They are there to protect dev teams from implementing misunderstandings of tickets. In a way a good Product Manager should wear a QA hat themselves, but I've seen fewer good PMs than good QAs....

Yup - I want to echo that the best PM I worked for also did quite a bit of QA work himself.

A deep understanding of the direct user experience of working with your product is a really valuable thing to have if you want to make users actually like the product. There's a WORLD of difference between the user experience that is mocked in a tool like figma or written down in a system like jira, and the actual live result.

As an aside, some of the most impactful engineers I've worked with also manually interact with the product incredibly often during development. If you automate a test too early with e2e tooling, you miss out on the wisdom gained by having to click through a feature 100+ times during development. Personally doing it exposes all sorts of rough edges and pain spots that your users are going to feel. Automating it makes you numb to them instead. It's a difficult balance.

Reminds me of how often I've felt a little envious as a dev of how much influence QA people had on effective specification. Whenever a spec appears a little ambiguous (or contradictory) the QA person becomes judge and their decisions effectively become law.

yes - devs are great at coding so get them to write the tests and then I, a good tester, (not to be confused with QA) can work with them on what are good tests to write. With this in place I can confidently test to find the edge cases, usability issues etc And when I find them we can analyze how the issue could have been caught sooner

Coz while devs with specialties usually get paid more than a generalist, for some reason testing as a specialty means getting a pay cut and a loss in respect and stature.

Hence my username.

I wouldnt ever sell myself as a test automation engineer but whenever i join a project the number one most broken technical issue in need of fixing is nearly always test automation.

I typically brand this work as architecture (and to be fair there is overlap) and try to build infra and tooling less skilled devs can use to write spec-matching tests.

Sadly if i called it test automation i'd have to take a pay cut and get paid less than those less skilled devs who need to be trained to do TDD.

I think there are 3 'kinds' of QA who are not really interchangeable as their skillsets don't really overlap.

- Manual testers who don't know how to code at all, or at least arent' good enough to task them with writing code

- People who write automated tests (who might or might not also do manual testing)

- People writing test automation tools, managing and desigining Test infra etc. - these people are regular engineers and engineering skillsets. I don't think there's generally a difference in treatment or compensation, but I also don't really consider this 'QA work'

As for QA getting paid less - I don't agree with this notion, but I see why it happens. Imo and ideal QA would be someone, who's just as skilled in most stuff as a dev (except does something a bit different), has the same level of responsibility and capacity for autonomy - in exchange I'd argue they deserve the same recognition and compensation. And not giving them that leads to the best and brightest leaving for other roles.

I think it's amazing when one gets to work with great QA, and can rest easy that anything they make will get tested properly, and you get high quality bug reports, and bugs don't come back from the field.

Also it bears mentioning, that it's self-evident to me, but might not be self-evident to everyone, that devs should be expected to do a baseline level of QA work themselves - they should verify the feature is generally working well and write a couple tests to make sure this is indeed the case (which means they have to be generally aware how to write decent tests).

> A lot of QA have expressed that devs 'look down' on them. I can't comment on that, but the signal-to-noise ratio of bug tickets is so low, that often it's you have to do their job and repeat everything as well.

When I was a lead, I pulled everyone, (QA, devs, and managers) into a meeting and made a presentation called "No Guessing Games". I started with an ambiguous ticket with truncated logs...

And then in the middle I basically explained what the division of labor is: QA is responsible for finding bugs and clearly communicating what the bug is. Bugs were not to be sent to development until they clearly explained the problem. (I also explained what the exceptions were, because the rule only works about 99.9% of the time.)

(I also pointed out that dev had to keep QA honest and not waste more than an hour figuring out how to reproduce a bug.)

The problem was solved!

Communicating a bug clearly is testing/QA 101

In my experience, I find that management doesn't understand this, or otherwise thinks it's an okay compromise. This usually comes with the organization hiring testers with a low bar, "sink or swim" approach.

Having worked with both good and bad QA...

The biggest determinant is company culture and treating QA as an integral part of the team, and hiring QA that understands the expectation thereof. In addition, having regular 1:1s both with the TL and EM to help them keep integrated with the team, provide training and development, and make sure they're getting the environment in which they can be good QA.

And work to onboard bad QA just as we would a developer who is not able to meet expectations.

I used to work with a QA person who really drove me nuts. They would misunderstand the point of a feature, and then write pages and pages of misguided commentary about what they saw when trying to test it. We'd repeat this a few times for every release.

This forced me to start making my feature proposals as small as possible. I would defensively document everything, and sprinkle in little summaries to make things as clear as possible. I started writing scripts to help isolate the new behavior during testing.

...eventually I realized that this person was somehow the best QA person I'd ever worked with.

how did misunderstanding a feature and writing pages on it help, not sure I follow the logic of why this made them a good QA person? Do you mean the features were not written well and so writing code for them was going to produce errors?

In order to avoid the endless cycle with the QA person, I started doing this:

> This forced me to start making my feature proposals as small as possible. I would defensively document everything, and sprinkle in little summaries to make things as clear as possible. I started writing scripts to help isolate the new behavior during testing.

Which is what I should have been doing in the first place!

If a QA person (presumably familiar with the product) misunderstands the point of a feature how do you suppose most users are going to fare with it?

It's a very clear signal that something is wrong with either how the feature was specified or how it was implemented. Maybe both.

I took GPs meaning that the QA person in question sucked, but them being the best meant the other QA folks they've worked with were even worse.

Let's call the person in question Alex. Having to make every new feature Alex-proof made all of the engineers better.

Did it? Sounds like making things "Alex proof" may have involved a large amount of over-engineering and over-documenting.

That's not at all what they meant. They meant they ended up raising their own quality bar tremendously because the QA person represented a ~P5 user, not a P50 or P95 user, and had to design around misuse & sad path instead of happy path, and doing so is actually a good quality in a QA.

It's possible but I'd guess they are probably not worse than the average user.

I worked with someone a little while ago that tended to do this; point out things that weren't really related to the ticket. And I was happy with their work. I think the main thing to remember is that the following are two different things

- Understanding what is important to / related to the functionality of a given ticket

- Thoroughly testing what is important to / related to the functionality of a given ticket

Sure, the first one can waste some time by causing discussion of things that don't matter. But being REALLY good at the second one can mean far less bugs slip through.

Most of the time QA should be talking about those things to the PM, and the PM should get the hint that the requirements needed to be more clear.

An under-specified ticket is something thrown over the fence to Dev/QA just like a lazy, bug-ridden feature is thrown over the fence to QA.

This does require everyone to be acting honestly to not have to belabor the obvious stuff for every ticket ('page should load', 'required message should show', etc.). Naturally, what is 'obvious' is also team/product specific.

I think noticing other bugs that aren't related to the ticket at hand is actually a good thing. That's how you notice them, by "being in the area" anyway.

What many QAs can't do / for me separates the good and the not so good ones, is that they actually understand when they're not related and just report them as separate bugs to be tackled independently instead of starting long discussions on the current ticket at hand.

so, QA should be noticing that the testers are raising tickets like this and step in and give the testers some guidance on what/how they are testing I've worked with a clients test team who were not given any training on the system so they were raising bugs like spam clicking button 100 times, quickly resizing window 30 times, pasting War and Peace.. gave them some training and direction and they started to find problems that actual users would be finding

I didn't mean reporting things that you wouldn't consider a bug and just close. FWIW tho, "Pasting War and Peace" is actually a good test case. While it is unlikely you need to support that size in your inputs, testing such extremes is still valuable security testing. Quite a few things are security issues, even though regular users would never find them. Like permissions being applied in the UI only. Actual users wouldn't find out that the BE doesn't bother to actually check the permissions. But I damn well expect a QA person to verify that!

Was I meant though were actual problems / bugs in the area of the product that your ticket is about. But that weren't caused by your ticket / have nothing to do with that ticket directly.

Like to make an example, say you're adding a new field to your user onboarding that asks them what their role is so that you can show a better tailored version of your onboarding flows, focusing on functionality that is likely to be useful for you in your role. While testing that, the QA person notices a bug in one of the actual pieces of functionality that's part of said onboarding flow.

A good QA understands and can distinguish what is a pre-existing bug and what isn't and report it separately, making the overall product better, while not wasting time on the ticket at hand.

Ha, that's certainly a way to build things fool-proof.

There’s definitely a bimodal distribution of QA people for capability. The good ones are great. The bad ones infuriating.

The lack of respect and commensurate compensation at a lot of companies doesn't help. QA is often viewed as something requiring less talent and often offshored which layers communication barriers on top of everything. I've met QA people with decent engineering skills that end up having the most knowledge about the application works in practice. Tell them a proposed change and they'll point out how it could go wrong or cause issues from a customer perspective.

This 100%

Companies think QA is shit, so they hire shit QA, and they get shit QA results.

Then they get rid of QA, and then the devs get pissed because now support and dev has turned to QA and customers are wondering how the hell certain bugs got out the door.

Yeah and then we started expecting them to code. Which has not gone well. And the thing is if you have the suspicious mind of a top rate QA person and you can code well, you’re already 2/3 of the way to being a security consultant or a Red Shirt engineer and doubling your salary.

This is why your duty in engineering is to drag QA into every specification conversation as early as possible so that they can display that body of knowledge.

Yes. The best QA people are gold. Infuriating at times, but gold.

> end up having the most knowledge about the application works in practice

The best I've worked with had this quality, and were fearless advocates for the end-user. They kept everyone honest.

I was at a company once where they were talking about trying to do a rewrite of an existing tool because the original engineers were gone. But the requirements docs were insufficient to reach feature parity, so they weren’t sure how to proceed. Once I got the QA lead talking they realized he had the entire spec in his head. Complete with corner cases.

The problem is usually in the company culture and hiring process.

Are the QA people & team treated like partners, first class citizens, and screened well the way you would an SWE?

Or are they treated like inferior replaceable cogs, resourced from a 3rd party consulting body shop with high turnover?

You get what you hire for.

We hired a guy with an English Lit degree as QA. He was super smart, and really self-motivated. He learned full-stack dev, and wrote a fcking amazing dashboard and test config wizard in like half a year. (This was before AI)

People at that point were complaining about tests being hard to run for YEARS.

He then left for a dev role at another company in a short time.

And “they had huge impact & left quickly” is actually a good outcome right!

Better than underhiring to set the whole endeavor up for failure

Why would you upskill as a QA when you can become a dev? Every single QA person I know only became a QA as a stepping stone. That's now it's seen.

Companies don't care about QA, so of course you don't see any QA wizards anymore.

Necessary but insufficient. On several projects where I was the lead I started honoring the QA lead's attempts at vetoing a release. I was willing to explain why I thought changes we had made answered any concerns that QA had, or did not, but if the lead said No then I wasn't going to push a release.

If you're consistent about it, you can restore a sizable chunk of power to the QA team, just by respecting that No. With three people 'in charge' of the project instead of two, you get restoration of Checks and Balances. Once in a while Product and QA came to me to ask us to stop some nonsense. Occasionally Product and Dev wanted to know why things were slipping past QA. But 2/3rds of the interventions were QA and Dev telling Product to knock something off because it was destroying Quality and/or morale.

God do I miss QA and being able to go 2 against one versus the person whose job description requires them to be good at arguing for bullshit.