Hacker News Reader

Comment by mobilene | Hacker News Reader

I've yet to see the automated test suite that replaces a skilled, sapient, human, functional tester. The automation takes away the drudgery of repeating tests, but it takes a skilled human to figure out what risks are in the code and figure out coverage to determine whether those risks are realized. If you have developers write good unit and integration tests, and build their work to their local environment to make sure it basically works, you avoid most of the "throw it over the wall" mentality. You also deliver way fewer needles in the haystack, if you will. Testers are free to look for more impactful bugs manually, and then to automate coverage as appropriate.

View on HN

There's a parallel between dev and QA here.

Writing code is generally easy. Figuring out what code to write is hard. I can spend 80+% of my day thinking about an overall system, what things are important, and how to design things so that they both work and don't prevent change later. Then sitting down and writing the code to do that is (usually) the easy part.

Writing automated tests is generally easy. Figuring out what tests to write is hard. Does the code break on normal cases? Does it break on edge cases? Does it handle all the known use cases? Does the implementation actually achieve what the client is asking for? Does what the client asked for actually make sense?

You can't just automate away testing. Automated testing (unit, functional, system, etc) is a power multiplier for both dev and QA... not a total solution.

There is even more to it than that.

Business thinks that you can do test automation in a vacuum without any domain knowledge and without knowing the system.

You always need people to understand the system, to understand the automation and to keep domain knowledge in house.

If your whole QA team leaves and you have automation (or even documentation) - you are still in deep shit because hiring new people and training them on the system will still be needed and only then they can learn to run the automation. They still need to spend time reading documentation and understanding it, because that something is written down or documented does not mean other person has the same understanding of it and the same reference framework to use that documentation.

Well you can hire some new people and let them run automation built by previous team but ... good luck with that.

I think you're pretty on point here; today human nuance is needed to decide WHAT to test as well as HOW MUCH to test. It's also useful for executing some types of tests too (which is why we support both automation, and human testing). IMHO Unit and integration tests should be a given. Human testers should be used with the highest leverage where possible. Being a functional-tester though, today is partly limited by what tooling is accessible to you - which we want to fix.

I've seen them used to do a decent smoke test to allow the QA people to make forward progress on other testing. I worked at a place that had a large number of computers running some testing software (robo something?) that ran through all the mainline scenarios. They constantly added new scenarios so the QA folks were working on much deeper flows until those too became automated.

Totally agree that automation will never fully replace the value of human-powered testing. (Though it is great for the rote regression-testing stuff. The "drudgery", as you put it.)

Isn't the problem with relying too much on unit and integration tests that they don't consider the end-to-end app experience? (Which, in my mind, is what ultimately matters when customers think of "quality".)

IMHO, yep - it's a balance, but the great thing is they can be quick to run, and run locally-easily; which is great for developers to get fast-feedback. Unit testing is unlikely to catch all end-user level issues though; traditional automation too, which is why human-testing is still valuable today.

They're not claiming people are not needed.

They say that instead of being developers writing codee, they should be non technical product managers using no-code.

Company from the article is not claiming that but all the business people are talking about it.

They would like that "test-automation" would mean instead of 2 QA engineers now you need 1 doing 2x more.

Sad reality is that with "test-automation" you actually still need those 2 QA engineers and maybe part time dev to help them out. The upside would be that those QA people would spend less time repeating clicks and improve quality of checks.

Reply on news.ycombinator.com