“Don’t mock what you don't own” in 5 minutes (2022) | Hacker News Reader

“Don’t mock what you don't own” in 5 minutes (2022)

hynek.me

42 pointsJNRowe19 comments

It would be great if every complex or volatile third-party library came with its own mock and an abstract interface to which they both conform. Including pieces of language’s standard library itself.

It makes so much more sense this way:

- users don’t have to duplicate effort of writing more or less same mock/stub of the same library

- mock is forced to be up to date, because the library’s maintainers probably pay more attention to their release notes than you do

- no one knows the best choice of mocking approach/DSL than the library’s authors

- the adoption of inversion of control / dependency injection and comprehensive testing could be so much wider if writing a test didn’t require first solving an extra problem if how to mock something with complicated API and if it worth solving in the first place

this is what we do internally at $dayJob. the gradle java-test-fixtures plugin (https://docs.gradle.org/current/userguide/java_testing.html#...) creates a dedicated source set and wires up artifact publishing with a `test-fixtures` classifier. consuming a libraries fixtures is as simple as:

    // pull in some-library
    implementation("some-library-coordinates")
    
    // use its provided fixtures in tests
    testImplementation(testFixtures("some-library-coordinates"))

wish it was more common in the ecosystem, but it seems most folks aren't aware of it.

I’m not sure this is good advice. I prefer to test as much of the stack as possible. The most common mistake I see these days is people testing too much in isolation, which leads to a false sense of safety.

If you care about being alerted when your dependencies break, writing only the kind of tests described in the article is risky. You’ve removed those dependencies from your test suite. If a minor library update changes `.json()` to `.parse(format="json")`, and you assumed they followed semver but they didn’t: you’ll find out after deployment.

Ah, but you use static typing? Great! That’ll catch some API changes. But if you discover an API changed without warning (because you thought nobody would ever do that) you’re on your own again. I suggest using a nice HTTP recording/replay library for your tests so you can adapt easily (without making live HTTP calls in your tests, which would be way too flaky, even if feasible).

I stopped worrying long ago about what is or isn’t “real” unit testing. I test as much of the software stack as I can. If a test covers too many abstraction layers at once, I split it into lower- and higher-level cases. These days, I prefer fewer “poorly” factored tests that cover many real layers of the code over countless razor-thin unit tests that only check whether a loop was implemented correctly. While risking that the whole system doesn’t work together. Because by the time you get to write your system/integration/whatever tests, you’re already exhausted from writing and refactoring all those near-pointless micro-tests.

> I’m not sure this is good advice. I prefer to test as much of the stack as possible. The most common mistake I see these days is people testing too much in isolation, which leads to a false sense of safety.

You make it sounds as if the article would argue for test isolation which it emphatically doesn't. It in fact even links out to the Mock Hell talk.

Every mock makes the test suite less meaningful and the question the article is trying to answer is how to minimize the damage the mocks do to your software if you actually need them.

But ultimately it suggests this test; which only tests an empty loop?

  def test_empty_drc():
      drc = Mock(
          spec_set=DockerRegistryClient,
          get_repos=lambda: []
      )

      assert {} == get_repos_w_tags_drc(drc)

Maybe it's just a poor example to make the point. I personally think it's the wrong point to make. I would argue: don't mock anything _at all_ – unless you absolutely have to. And if you have to mock, by all means mock code you don't own, as far _down_ the stack as possible. And only mock your own code if it significantly reduces the amount of test code you have to write and maintain.

I would not write the test from the article in the way presented. I would capture the actual HTTP responses and replay those in my tests. It is a completely different approach.

Yes, because it's showing how the simplest-possible mock already gets ugly if you don't follow the advice. Making the example more complicated would dilute the point it's making.

The question of "when to mock" is very interesting and dear to my heart, but it's not the question this article is trying to answer.

My take on unit tests is that they happen "automatically" for the things that actually need unit testing. Consider a load shedder, a parser, or an implementation of an algorithm. Most devs will naturally write a unit test for anything with complicated logic just because it's easier to test complicated things in isolation.

And better yet, follow the tangential advice and use high fidelity fakes (I usually don't quite go all the way to verified fakes/contract tests, but they're a good idea if you have the time).

Real object where practical, fakes otherwise, mocks only as required for exceptional circumstances.

Works great in interconnected monorepos where you can provide high fidelity fakes of your services for other integrating teams to use in their tests. Often our service fakes are literally just the real service wrapped with an injected fake DB/store.

Fakes are better than mocks if only for the 100x superior debugging experience.

Something that I find amusing in the Java community is that a good number of senior developers, with anything from 5-20 years of experience, who do 'tdd' have never heard of the concept of test doubles and religiously think that a class must be tested in complete isolation mocking everything else.

The saddest one I saw was a team trying to do functional programming (with Spring). The tech lead was a bit flummoxed when I asked why mocks are not used in functional languages and continued to think that 'mocking functions' is the correct way to do TDD.

Tests knowing about implementation details and testing the implementation details (which is the case 99.999% of the time if you use mocks) is more common than not. Even when the main value of automated testing is being able to change those very implementation details, that you now cannot do.

A whole bunch of work spent for no benefit or negative benefit is pretty common.

I use Java and Spring extensively. If I am in a lead role and have say over the code base I won't allow mocking frameworks to be used in tests. If you want a good way to shine a light on poorly structured code, disallow mocking frameworks.

Java added the Funcional interface in v8 making it quite easy to code to an interface and yet in Spring everyone's go to is just to slap a @Component on a concrete class.

Polite way here is similar to inversion of control. It's a good idea but you need a real thing test somewhere but that can be running the program in its entirety, or using the facade end to end against the real interface for a spot check.

Ok but the test is now not covering the parts that are actually going to break. And if you're only testing the parts above the nice business-friendly interface, you probably don't need mocks at all.

I'm confused. If I'm allowed to mock what I don't own, but only if it has a good API that's well-adapted to my use case, then "don't mock what you don't own" must have nothing to do with ownership.

It sounds like it's more about wrapping awkward APIs instead of calling them directly. Yes, this is good advice! But it only tangentially has anything to do with mocking.

This is very similar to Jose Valim's "Mocks and explicit contracts" from 2015, down to using a Twitter client as example.

https://dashbit.co/blog/mocks-and-explicit-contracts

Why I don't mock (2014): https://news.ycombinator.com/item?id=7809402

"Well, it's impolite, isn't it?"

I love Stripe for creating a CLI testing thing that even allows you to test webhooks.

(2022)

Reply on news.ycombinator.com