Hacker News Reader

Comment by josefx | Hacker News Reader

> It pains me when I see developers testing their functionality end to end, manually

On the other hand I know developers that only unit test. Hundreds of lines of mock code, dozens of tests. No documentation, no sample configuration for the final application and dozens of data races because the unit tests don't cover multi threading. But hey an unusable buggy mess is a net positive as long as it makes the test coverage statistic happy and that is one of those things reported to management.

While a bit exaggerated one of the downsides of unit tests is that they quickly become part of a metric without regards for when they ad value.

View on HN

> Hundreds of lines of mock code

Mocking should be kept to a minimum and ideally avoided.

Mocking is most useful when adding tests to legacy code, but new code should be designed so it can be tested without the need for mocking.

Also can be pretty handy when there's hardware that needs something very specific to happen to work properly. Mocks can give you (some) confidence that when you poke the I2C bus, you won't hang it up.

But it's very easy to overdo it and end up testing irrelevant implementation details instead.

Mocking own code should be avoided if possible, but it does not apply to external resources like database, http responses etc.

I don't think it's ever a good idea to mock a database. Invest in a test framework that can performantly spin up a fresh database instance and throw it away at the end of the test run instead.

Mocking outbound HTTP calls is worthwhile, but thankfully most testing frameworks I've used (at least in Python world) make that pretty easy.

I mock database by using sqlite in-memory database. It is not 100% compatible with my production database but still god enough for most of my tests. This way I have fresh database for each unit test and the tests are still fast.

Hence why you need automated integration tests, where possible. Especially for regressions.

They tend to have much higher value than pure unit tests.

I agree that automated integration tests typically offer higher ROI than equivalent unit tests written with mocks, because mock code is so difficult to write and fragile.

I disagree with your assertion that integration tests offer higher value than unit tests in general. Don't judge all unit tests by mock code.

I will counter that integration tests tend to have a much bigger footprint: one test covers a lot more ground and so per line of code will catch more things.

But, the failures from integration tests are often a lot harder to read. Even half-good unit tests tend to show exactly where the error is, and make the fix cycle much shorter. It is also much harder to write exhaustive integration tests for all of the inputs that might happen.

I, as a long-term qa-ish person (I usually write testing systems) see real value in having a mixture of both system, integration (multiple systems), and unit tests, as well as end-to-end manual tests. Anything big enough needs all of these.

This is the point. Both are tools. They're both needed.

If errors from your integration tests are hard to understand, that just means you need to improve the debugging information emitted by your software such that your developer can debug it properly using it, no?

TBH mock code just sucks to write, no matter how good your unit tests are.

That is one of the big benefits I have found from writing code while planning to write unit tests for it (even if you don't write all of them). You will generally avoid patterns that will require a lot of mocks to test, which generally make that code easier to refactor and reason about. If you are able to make big chunks of your code into "pure" functions, they are now really easy to test.

A classic approach is that if you have to make some complicated API call with a bunch of arguments, you can put all the logic wrangling the arguments into a pure helper function and unit test the helper.

Agree.

I use strategy pattern with Java functional interfaces to avoid using magic mocking frameworks. That way, my mocks are just trivial impls of the plugin parts.

How do you accomplish that? I often end up in a situation where mocks are needed (like in the article).

Often times, you will have chunks of code that are either creating input for what would go into an interface/mock, or handling the output of it. You could pull those bits, even if they are relatively simple (maybe only a few lines, and only used in that one spot), and extract those out to a pure function to sidestep the mock.

Sometimes you can't really avoid it. In those cases, I would judge how much of the effort is testing your code, and how much is testing your ability to write mocks. Unless it is a really important functionality, or something complex enough that it would be easy to get wrong, I wouldn't bother with most tests that need extensive mocking. Instead handle those later with integration / e2e tests.

Also agree.

I've seen code so mocked it was dubious that it could be testing anything, accurately at least.

Setting up sample data is the biggest pain for mocking in my experience. But it's true however you're testing.

> and that is one of those things reported to management.

Yeah unit testing as I see it is just optimising for the success of middle management, at the expense of both the organisation's goals, and the developers.

You are making predictability the only focus, because that's the metric which middle management is evaluated by. At the expense of performance, end product quality, and work satisfaction. "It was done in exactly the excruciatingly slow and mind numbing boring way, and was exactly as shitty as designed" = success for management.

If you would try to increase performance, end product usability, and/or work satisfaction, you would have to put the predictability at risk, which means that the success of the middle manager will not be the highest priority at all cost anymore.

Your comment is downvoted perhaps because it sounds so cynical, but after thinking about it I agree to a certain extent. Throughout my career I've noticed the biggest pushers for more unit testing are usually middle managers who love to see that coverage number goes up since it's an observable metric, but nuanced opinions like "more unit tests are not always good and can be detrimental" are generally not welcome.

yep. I've worked at places with unit tests/e2e/and static typing. And I've worked at places with zero unit tests/no e2e/dynamic typing.

The push for more testing and static typing (to degree) was always managers. They trade predictability for momentum.

The reality in both situations is that bugs happen. I've not noticed a difference in frequency of "critical" bugs in either scenario. The only difference is that at the no unit test/no e2e places, getting your code into production was trivial. Which always meant when bugs happen, they are fixed much faster than the unit test-everything place. We're talking minutes vs. days here. Because at a risk-adverse business (that is, any place more than a few hundred people), you have multiple stakeholders that have to sign off if you want to go take a piss in the bathroom.

While I totally agree that process can go nuts and get in the way, I will disagree that testing always makes things slower. Once you are past the prototyping phase, and get into actual production phase (some projects never make this turn) some testing infrastructure can absolutely make thing faster. Rather than having to deal with all of the bugs that your fellow developers are just dumping into the main branch, they can help weed out some of the bugs before others have to deal with them.

> Hundreds of lines of mock code

No wonder they are not getting value out of unit tests.

https://blog.metaobject.com/2014/05/why-i-don-mock.html

Reply on news.ycombinator.com