Hacker News Reader

Comment by gwbas1c | Hacker News Reader

Just remember that automated tests only prove that the tests pass; they don't prove that your product works. You still need some amount of manual testing.

What automated tests do, though, is make it significantly easier to test. You will find bugs faster, and dramatically shorten the amount of manual testing that you need to do.

IMO, the best way to learn TDD is via experience. You need to work with a team that's willing to support a junior engineer. From a senior engineer's perspective, the amount of handholding that some junior developers need through writing decent unit tests is quite time consuming.

Thus, if I were you, I'd just be selective about who I work for. I'd look for a team that's willing to be patient while I learn, and that has enough senior developers that I can learn by following a good example. A team that wants a lot of experience in this or that is probably not able to allow you the time needed to learn.

View on HN

There is nothing more reliable about manual testing than automated testing. If you are finding that your automated tests are often passing when the product is broken then your approach to testing likely needs more work, this is common when people rely too heavily on unit testing.

A good approach to avoiding this situation is, what is that manual test you do to make sure stuff works? automated it.

> Just remember that automated tests only prove that the tests pass;

They also are code and therefore may contain bugs, impose additional maintenance overhead, and have all the other pros and cons that code has.

Also, it's possible to over-test in the same way it's possible to overengineer.

> Just remember that automated tests only prove that the tests pass; they don't prove that your product works.

Doesn't this show that either your test case is not product requirements based or the design is not aligned with the product requirements?

It's true, but making sure that the tests perfectly cover the product requirements, and making sure that the specified product requirements actually match the expectations of the product are both barely solvable problems for any sufficiently complex product.

Trying to keep pace with most markets, it's hard for software developers to maintain detailed enough requirements to even hope to ever achieve exhaustive automated testing.

Which design? This weeks, or next months?

> You still need some amount of manual testing

What is it that you're testing manually that you can't automate?

That a human agrees that the product works correctly. These can be silly things like observing that buttons overlap, or observing misspellings.

"What is it that you're testing manually that you can't automate?" I can't automate what I can't predict.

Simple example: Would you get in a robotic car that only had automated testing? What about a robotic flying car?

Real example: 20 years ago, I bought a modem that had an "obvious" problem when calling a dial-up BBS. It didn't flush the buffer appropriately, leaving me with an incomplete screen of text.

I'm sure the modem passed all of their automated tests! But, did whoever wrote the automated tests predict that the buffer should flush if no data came within a very small amount of time? Nope!

Another example: Have you recently used Netflix on Android with Chromecast? It breaks in a lot of corner cases when you put your phone in your pocket. Netflix proudly laid off their manual testers and only does automated testing. Can you really automate every aspect of the Chromecast in a test harness? I bet you can come close, but not achieve 100%.

My tests usually involve common expected/actual cases. Manual testing can uncover "unique" scenarios where my code can break.

Manual testing can discover bugs. But the first step toward debugging should be writing an automated test to exercise the manually discovered bug, imho.

That said, bug-driven testing is a separate issue from TDD.

Any advice on finding such companies?

Treat your interview as a two-way street. You should interview the company as much as they are interviewing you.

Ask questions about their testing approach. Remember, no company is perfect. Often, small startups decide it's in their best interest to rely on manual testing and accept that their products are buggy. This can be an opportunity if you're able to clean things up, though.

If a company is honest, and takes the time to answer your questions, then it's probably a good sign. If the company thinks you should be grateful for a job and grovel for whatever crumbs they drop, RUN!

Remember, though, no company is perfect. It's better to work for a company that has an imperfect TDD then one that treats you like dirt.

It's hard. Because everyone claims to be doing TDD, just asking in the interview "do you do TDD" is pointless.

Example. Last year I consulted at an IC company, building a huge "moon-shot" chip. You might expect TDD to be an absolute certainty in such an operation. Not so, individuals had all sorts of justifications for not doing TDD.

Here is one idea. In the technical interviews, when the inevitable "and do you have any questions for me?" moment comes up, ask them to talk about their best test.

If they have a good answer, it's likely you've found somewhere that genuinely values TDD.

I agree with this. I've interviewed a lot of people recently for a senior developer role. The difference in people's ideas of what TDD or testing in general, is quite marked.

I love it when people ask me about how we test. I can tell you the pains we've had with tying our tests too tightly with implementation. :D

Reply on news.ycombinator.com