Hacker News Reader

Comment by stephen | Hacker News Reader

Agreed with the articles, mocks are very tied to implementation details. I almost always prefer state-based testing:

https://martinfowler.com/articles/mocksArentStubs.html

As "observable state goes from a to b" is much closer to a business/functional requirement that will still/always be true, regardless of refactorings.

Refactoring in codebases with state-based tests is a pleasure; in codebases with mock-based tests it's tedious, constantly updating tests when no semantic behavior was supposed to change.

Also, mocking via module hacks like in the article (and in the JS world) is scary; modules are basically global variables so it's a very coarse grained slice point. Dependency injection is almost always better.

View on HN

Dependency injection leads to elaborate, brittle fixtures that become more elaborate and arcane over time.

And the sunk cost fallacy has people working for hours (and I’ve witnessed pairs spend two days, that’s over three man days!) trying to maintain them.

I use mocks to arrange state A without going through all the business rules involved in creating state A. But you have to expose the states to do it, which can make for pretty descriptive code but is outside of some people’s experience and so they resist it.

Earlier tests have already verified all routes to State A, the next batch of test now takes it as a given. This controls test explosion by utilizing transitivity. A->B B->C implies A->C. Tons of unit tests for A->B and B->C and then you only need a couple of functional tests (say, one negative and positive) test of A->C. You’re just checking the plumbing isn’t broken.

Otherwise, you get Cartesian products. You end up with elaborate, (often custom), mocks that couple all of the tests together in hard to maintain ways. You end up with tests that accidentally test the mocks/fixtures instead of the code.

Some of my current coworkers do this too. I don’t know where this pattern comes from. It’s often easier to replace their two page fixtures with two or three lines of mocks per test. It’s often almost the same amount of code, but so what, each test can be read. Each test can be run individually, and behavioral changes to the code due to urges affect mostly the tests you would expect. Those tests can be fixed, rewritten, or just removed as they disagree with the new requirements.

The absolute worst is fixtures with asynchronous code. Those break constantly, and often invisibly. What’s the point of a fire alarm if the damned thing doesn’t work? It’s almost worse than nothing at all.

> Dependency injection leads to elaborate, brittle fixtures

I believe that can happen, but personally (N=1) haven't seen it. If anything, (well-done) DI is supposed to prevent that, b/c fixtures are isolated instead of depending on random global behavior.

That said, I also don't like Spring/Guice/Dagger auto-wiring DI (somewhat guessing that is what caused your headaches), and instead just create an "AppContext" type with all of the applications "singletons" and pass that around.

Granted, it's still global-ish, so maybe I'm cheating, but its nicer IMO than module hacks.

> I use mocks to arrange state A without going through > all the business rules involved in creating state A

I like tests being able to immediately jump to state A, but fwiw don't see why mocks would be needed to do so.

I do agree re avoiding test explosion/transitivity, and a few functional happy/sad plumbing tests, but again seems orthogonal to state/mock.

> You end up with elaborate, (often custom), mocks that > couple all of the tests together in hard to maintain ways

Totally agreed. Another -1 for mocks. :-) I.e. with state-based you should be able to test "end-to-end" (with state-based / in-memory stubs for your input/output data stores/etc.) without any of "oh right, copy/paste these 5 lines of 'when method X return result Y' for ~2-5 some mocked-out calls).

(And, to be pedantic, if you mean something other than 'when method X return result Y' for the term 'mock', then we're probably talking about different things.)

> That said, I also don't like Spring/Guice/Dagger auto-wiring DI (somewhat guessing that is what caused your headaches), and instead just create an "AppContext" type with all of the applications "singletons" and pass that around.

> Granted, it's still global-ish, so maybe I'm cheating, but its nicer IMO than module hacks.

I agree with you. I would like better support for (a) intersection types and (b) excluding global names.

With intersection types, you avoid the globalish nature, since you can say "this method takes a PersistentUserStore", "this method takes a LoggerInterface", and "the parent method takes a PersistentUserStore&LoggerInterface" which can be passed to both. Lo and behold, the fact that it's globalish disappears since at any given point, you only

Unfortunately, as with all things, PhpStorm does this exactly wrong - so that an intersection type is inspected as tho it were a union type and it gives me no particular benefit. Jetbrains seems less interested in helping me avoid bugs and focuses on helping me avoid reading the php.net manual. I appreciate it when I don't need to check out the manual, but I need to avoid bugs.

As for excluding global names - if there were some way to make it illegal to say `new Foo`, except in the top of the code, for certain values of "Foo", it would be a serious help in keeping my tests of business logic free of implementation details. I'm not entirely sure how to restrict those "certain values of Foo" though.

> The absolute worst is fixtures with asynchronous code

I concur that setting up a test for async code is a complete pain in the arse. It's extra fun when the caller doesn't get a reply, so there's no response that you can assert your as valid or invalid. It's extra, extra fun when the tests in your test suite need to run in parallel without stepping on each other.

Here's an example. Say you've got PipelineService123 that receives a message with a chunk of data, transforms that data, and sends it elsewhere. Try thinking about dynamically setting an instance of PipelineService123, wiring up an input message sender, giving that sender the dynamic address of the instance of PipelineService123, wiring up an out message receiver, making sure the instance of PipelineService123 knows how to reach the receiver, all to test whether or not everything is wired up correctly & that the data is being sent, transformed, and received properly, running this test in parallel with all of the other test cases, and keeping it reasonably easy to reason about. Good luck with that! I'm not sure even I even stayed on top of my attempt to describe it :P

I've noticed that my tests are a lot less tortured when the language supports async/await.

But I do wonder if await should be the default behavior, since most async code is still sequential within the operation, at least until we start composing async operations. In which case having the keywords point out the composition probably improves reading comprehension.

Just curious : which language do you work with ?

In my previous work I had to work on a big asynchronous codebase in JavaScript and I had zero issues with asynchronicity at all, so maybe it's an ecosystem/language dependent issue ?

Elixir. Are you running your tests in parallel in the same process?

It depends on what you mean by parallel: nodeJs is single-threaded, so every CPU-bound task runs sequentially. But all the asynchronous tests are run concurrently by the test runner (Mocha[1] in my case).

[1] https://mochajs.org/

I don't see how you decided that DI requires some kind of special fixtures that are more complex than mocks.

Mocks are generalized fixtures—just a reusable way to create them. And you can use them with DI. Only the method of delivery is changed, from an implicit backdoor to an explicit argument.

DI puts new requirements on production code and tests that aren't related to a specific mock, that much is true. At least if there's no automatic way to create ‘default’ dependencies.

That was an empirical statement. Inasmuch as it was a value judgement, it was a condemnation of hand-rolling your own mocks. Which people seem to do with an alarming regularity, and with a fairly consistently appalling degree of skill.

And not junior developers.

> I use mocks to arrange state A without going through all the business rules involved in creating state A.

An issue is that unless the types are really nailed down you can create easily create invalid or incomplete states, or create a state which is correct at one point then becomes invalid / incomplete at an other.

> Earlier tests have already verified all routes to State A

Unless they also assert the complete state at that point and that assertion is tied into the next state[0] then what they've verified is a series of steps.

Seems risky to assume it matches what an unrelated tests starts with, unless you've implemented state restoration, at which point you'd have to add cross-test dependencies such that the result of the tests validating section A can be saved and restore for every test needing a state A to test section B.

[0] that is, ∅ -> A tests end with an identity check for state A (that the system entire has no more and no less than expected) and that A -> B tests start with one, right after they've generated their fake state

> As "observable state goes from a to b"

You might also notice that this is equivalent to a function that accepts a and returns b. Which lets one use equivalence partitioning to plan the tests: https://en.wikipedia.org/wiki/Equivalence_partitioning

Except that with singletons, ‘context objects’ or similar quasi-global state, the effect might be non-isolated, and effects may be silently introduced that the calling code doesn't know about.

Even better than "observable state from a to b" is "a transform of value from a to b", no inspection and whitebox needed :D

Absolutely. The more functional we can make the code, typically the easier it is to test it based only on inputs and outputs.

Much of what I do is try to come up with architectures that model the logic in this way, and are still legible to my co workers.

A situation in which I've used mocking well: testing parsing of data coming over a serial port. I have had several situations to need to parse common data formats coming from hardware over a wire and mocking that data with recorded streams is super helpful for making sure your parser is working right.

It sounds to me like you've got two things: some hardware code, and some data code. You're not testing the hardware code, you are testing the data code.

If you split them into two places, you could get the same effect, with no mocks. And be more clear about what parts are tested and what aren't.

That is literally what I said I was doing.

Reply on news.ycombinator.com