Ask HN: How to approach testing in Golang? | Hacker News Reader

Ask HN: How to approach testing in Golang?

98 pointskc111635 comments

I am a bit lost when it comes to unit testing in Golang. When developing in Java and Javascript it is easy to mock out data and functions, make assertions and expectations such as making sure some function was called inside a function your testing. What are some things to make sure to test when unit testing Golang functions and what is a good strategy for mocking things out?

View on HN

Ha! There are maaaany opinions in this thread, but I will add mine.

I'm not a fan of writing stupid mocks. That doesn't give me much confidence, and spending time on tests is mostly about building ones confidence. I very much prefer running the program end-to-end with defined inputs, and checking its outputs.

Whether that's database, file on disk, or REST api. I write python nosetests that prepare the setup, run golang program, run the checks, and tear down the setup. Sadly, normally it's hard to get golang code coverage for end-to-end tests (aka integration tests). This is what I usually do:

https://blog.cloudflare.com/go-coverage-with-external-tests/

Take a look at the Golang source for some examples of table-driven tests. When you need mocking you can usually use interfaces and then define a struct in your test that satisfies the methods of the interface.

I'm currently co-authoring a book called Production Go which includes a chapter on testing. That chapter is available in the free sample: https://leanpub.com/productiongo/ and might cover some of your questions.

We make use of https://github.com/onsi/ginkgo and Gomega https://github.com/onsi/gomega for BDD testing with more expressive matchers. You might be familiar with some of these concepts from Cucumber/Jasmine/Mocha.

They get significant use in Cloud Foundry and in the Kubernetes e2e tests.

I appreciate a lot of people like the simplicity of the standard test library and if you're starting with Go development, it's not a bad idea to get used to that before exploring external dependencies.

Caveat: While not necessarily a "maintainer", I have been doing a bunch of maintenance on these projects the past few weeks.

Edit: Also we use counterfeiter https://github.com/maxbrunsfeld/counterfeiter for fakes.

I like to use a pattern like this: given a resource defined as an interface:

    type Database interface {
        Users() ([]*User, error)
    }

I like to create an explicitly-mocked version of it like this:

    type MockDatabase struct {
        UsersStub func() ([]*User, error)
    }

    func (m MockDatabase) Users() ([]*User, error) {
        if m.UsersStub == nil {
            panic("MockDatabase.UsersStub is nil")
        }
        return (m.UsersStub)()
    }

Then you can create the resources you need, and define how they work, at the top of your test:

    func TestSomething(t *testing.T) {
        db := MockDatabase{
            UsersStub: func() ([]*User, error) {
                ...
            }
        }

        // use db here
    }

This works well as long as the things you're testing take all the resources they need as parameters.

This is pretty much it, mocking is quite simple and straightforward. It's tedious but wrapping access to external resources (eg: cache, HTTP, database) is necessary for a sane Go-based application.

I also use that same pattern. I'm very tempted to write a tool to be called by `go generate` to automatically create that kind of mock.

Please don't. I mean, do, whatever makes you happy. There's at least 3 of these flying around in various stages of abandonment.

Here's one:

https://github.com/vektra/mockery

Looks like there's an official tool: https://github.com/golang/mock

Do any of those build this style of mock? I looked around a while back and didn't come across one.

edit: I looked again and found https://github.com/matryer/moq which seems similar

Design components based on interfaces. Use dependency injection, Testify [0] for asserts and mocks, and Mockery (based on Testify) [1] to generate mocks.

[0] https://github.com/stretchr/testify [1] https://github.com/vektra/mockery

I found the talk "Advanced Testing With Go" to be illuminating.

Video: https://www.youtube.com/watch?v=yszygk1cpEc

Slides: https://speakerdeck.com/mitchellh/advanced-testing-with-go

Java and co can generate logic at runtime, go cannot so you have to manually write mocks, or "generate" them with a third party executable.

You cannot mock anything if you don't use go interfaces at first place, there is no inheritance in go so you can't swap types unless they are interfaces or convertible.

given an interface Fooer :

    type Fooer interface {
       Do(int)int 
    }

and a method to test

    func Accept(f Fooer){
       f.Do(10)
    }

it's easy to write a mock

    type FooImpl struct {
      T testing.T
    }

    func(f FooImpl)Do(i int)int{
      f.T.Helper()
      if i!=10{
         f.T.Fail("i=10 expected")
      }
      return 0
    }

    // the test
    func TestAccept(t testing.T){
       Accept(FooImpl{T:t})
    }

Again, your methods have to accept interfaces at first place or you can't mock anything. With Go you'll often have to step back and do manually the things you took for granted in Java.

Only go maintainers can fix the "issue", by allowing the method definition on ad hoc structs, which is impossible right now.

Also, consider what you are really testing with mocks. Your implementation, or your mock? You can test without mocks..,

A mock is a fake. Just so we are using the same terminology I'll define my terms:

Fake: a test implementation of something that stands in for production code to solve certain testing problems.

A fake can implement an API and provide simplified behaviour. A good example might be a piece of code that stands in for a hardware device driver that you use when you don't have access to the hardware. Another very common use of a fake is to fake out a UI layer. There are lots of other common places for pure fakes.

There are 2 special kinds of fakes:

- Stub: An implementation that returns canned data.

- Mock: An implementation that encapsulates an expectation.

The most common type of mock is a mock that encapsulates the expectation that a certain function is called.

Sometimes you don't want to write a pure fake because it's more than you need. Most of the time you can adequately test what you want without talking directly to some inconvenient real API with a stub.

There are many good reasons to use pure fakes or stubs. Like I mentioned, it might be because hardware is involved. Sometimes it's because you are using some SaaS that you don't want to really access in your tests. Very often it's because the real service is just too slow (I'll need another post to describe why this is super important, but even if you don't agree I hope you will see that it's not the only reason to use a fake or a stub).

As soon as you use a fake or a stub, though, you leave a hole in your tests -- you have no way of knowing that the production code uses the real service in the production code. This is when you need a mock. You can go without testing it, but mocking it in this circumstance is almost always a better solution.

Of course, there is a school of thought that reaches for mocks first. This school of thought is often referred to as the "London School" because it is very popular (and probably originated from) several very famous people who work in London. The GOOSE book, which describes outside-in programming is probably the best description of this school of thought. I've met quite a few of the people who are standard bearers for this school of thought and they are very talented developers. Again, I would need another post to describe why I don't think it's the best technique to reach for first, but it's an area where reasonable people can disagree.

Outside-in and promiscuous mocking has several advantages -- especially when you are less experienced. It provides a framework for reasoning about how to do design. Personally, I am not a big fan in general, but there are times I use it -- I just replace the mocks with non-faked tests after the fact.

So, while I agree that one should have the mindset to use mocks as a technique of last resort, it's still a valuable tool in your arsenal.

>Outside-in and promiscuous mocking has several advantages -- especially when you are less experienced. It provides a framework for reasoning about how to do design.

I find that any additional couplings to your code makes the couplings you do have more painful to work with.

Some people think that by adding this kind of 'tight coupling' - low level mocks / unit tests - you feel that pain earlier on in the design process and that pain drives you to make a better, more loosely coupled code design.

Personally, I think that this approach is rather like forcing kids to smoke cigarettes every day to show them how much of a filthy dirty habit it is.

Very similar to cigarettes, the really nasty effects don't usually materialise until quite far down the road -- when you don't have much opportunity to deal with them.

However, I don't think it has to be like this. I sometimes call mocking "wish based design". Sometimes you don't really know what shape is going to be good. You know ahead of time that TDDing a solution is likely to churn a lot of time writing stuff you are going to ultimately throw away. You can spike a solution to get some more information, but there is often a lot of pressure to ship whatever you spiked, no matter how successful you were (or how bad your tests ended up being).

With a mocking approach you think to yourself, "If this were already written and I was using it, ideally how would it work?" You mock your wish and you write the use side of the code in your tests. Quite frequently this reveals most of your naivety and allows you to design what it is you need. At that point, you replace the mocks with real implementation. It's at this point where I differ from the main proponents of the style (or at least last time I talked to them, which is admittedly several years ago). I will TDD my implementation, removing the mocks as I go.

So, on reflection, I guess I use it in places where I would otherwise spike a solution. Again, for fairly junior people, it can be a great technique to help them understand where to start. Often I find that junior people can't do TDD because they simply can't envision what they are building. They will spike a solution, jam a couple of tests in to show that it's basically working and call it a day. A mocked solution often ends up exposing state in the places where it needs to be. This is frequently better than the spiked solution which is usually an encapsulated ball of mud.

As I said before, though, I use it as a technique of last resort, not a technique of first resort.

>However, I don't think it has to be like this. I sometimes call mocking "wish based design". Sometimes you don't really know what shape is going to be good.

That's why I write executable specs at a high level with my "wish based design", make that "test" pass and then refactor. Wish first -> code next -> only well designed code after that.

I don't feel the need to dive down another level and mock out real modules that are already covered by this high level test. That's just creating unnecessary work for myself and introducing a potential source of test bugs (where the mock doesn't match the reality).

> You can test without mocks

Can you elaborate, specifically on a unit-testing level?

> You can test without mocks...

There are some things that are implausible to test without injecting not-production-code. Many of these look like errors (bad inputs, expired sessions, etc.) that should be intelligently handled.

Check out https://github.com/stretchr/testify for more convenient assertions, as well as mocking support for implementing interfaces within your tests.

You cannot do ad-hoc mocking of arbitrary functions in Go. If you want to write component-level tests that mock out other components, then you need an interface boundry between those components. Or better yet, you just write tests at the network level, spinning up mock clients/servers on localhost.

I find vanilla Go testing (if res != exp { t.Fatal(...) }) much more versatile than testify. Testify did not cover some of my mocking use cases, and response on the repo was tepid, so I ditched it and was happier (this is a user POV, please don't come bashing me on not contributing to open source, etc.)

For mocks, I use GoMocks (https://github.com/golang/mock), again a "standard" lib of go. It has Expect syntax where you can assert that mocks were called in a certain way.

I use `go generate` to generate my mocks via interfaces. You should be comfortable with abstracting your specific implementation with interfaces. It is like the first must-do step to Go testing, since we cannot monkey patch or duck type stuff on the fly unlike in JS or Ruby... Second step is to generate mocks from the interface, and inject these dependencies into say your API HTTP Handler... Third step is to assert that your mocks were called in a certain way, and you can also craft different return values from your mocks, to test different scenarios. I also extended some of the gomock Matchers to be able to return different stuff on the fly for e.g in the case of row.Scan(&ptrToInt, &ptrToString), etc.

Ultimately I think this is the Golang ethos: Stick to the standard way of doing things, even if you are allergic to it at first. You will be happier.

Not a good example (if res != exp { t.Fatal}) is almost entirely require.Equal in testify (the message is logged that you specify, and the test is immediately stopped).

https://github.com/stretchr/testify/blob/master/require/requ...

I get a lot of the rationale behind the standard way ethos. I think this is just a bad example of it. Tests and lack of good OOB patterns for passing up error information being two of my primary issues with the "Go" way (and it only intensifies over time), but that's just my feelings and I get I'm in the minority (though Cheney seems to have a lot of the same issues I have with the standard error package https://dave.cheney.net/2016/04/27/dont-just-check-errors-ha...)

I don't fully grok this logic. So if Google adds a test assertion framework to the stdlib, then you would recommend using because it is now "Go approved"? In that event nothing has changed other than where the code is coming from.

You can't reasonable say that both methods are equal. Would you still recommend the (if res != exp { t.Fatal(...)}) in this case? If "No", then doesn't that mean you actually prefer more complex assertion primitives?

Do you only use "if" test conditions in C, C++, Java, Python? Why or why not? If "No", why is it acceptable to use more code and logic just because you switched to Go?

If you're comfortable with this style of testing, you can do it in Go using interfaces and code generation.

Counterfeiter hooks into `go generate`: https://github.com/maxbrunsfeld/counterfeiter

Here's an example project: https://github.com/andreasf/cf-mysql-plugin

If you want to make testing a bit less verbose, you can look into testify (https://github.com/stretchr/testify), it adds some helpful assertions (no more 'if err != nil ...').

If you are using interfaces with dependency injection, it is easy to mock the dependencies of whatever you are unit testing implementing the interfaces you have defined in your testing file.

A colleague of mine made a tutorial for GoMock, maybe its helpful: https://blog.codecentric.de/en/2017/08/gomock-tutorial/

It's pretty much the same as in other languages, so don't over think it. The main difference for you will be Go is compiled, so you have to think slightly differently when writing tests.

Here are a few don'ts:

- don't take interfaces as arguments unless it really makes sense, even if it makes testing easier

- don't use one of the tiny test helper libraries that basically just writes if statements for you

- don't start writing tests until you've already figured out how to organize your logging and errors

One pattern I found helpful was to write an adapter that uses the testing.T.Print* functions as the destination for log.Print* statements in my regular code so that log messages aren't jumbled when running tests in parallel.

1. Interfaces allow swapping out objects.

2. Mocking is a bad idea even in languages that support it well.

A better approach, when you do need to deal with fakes, is verified fakes: write two versions of an interface, and a test suite for that interface to ensure the fake acts the same as the real thing.

More here: https://codewithoutrules.com/2016/07/31/verified-fakes/

I have recently created a small project called rocket (https://github.com/joernlenoch/rocket) to provide myself with a simple dependency injection module for exactly this reason.

This is rather a starting point than a solution for your problem. But combined with some "fakers" and supplied testing frameworks this might remove some of the boilerplate.

The age of Docker means IMO you should avoid mocks. There's no reason not to have a full integration test against your real API especially because you can compile and run test binaries independently.

Not an apologist for unit tests, but I'll bite here: unit tests are still valuable when you have the capacity to run ful integration tests.

A unit test can run really fast, for one, it can cover future use cases that don't get covered in integration, it edge cases you don't expect to hit, but are still important to cover in case something happens.

I'm sure there are more, good reasons to write unit tests and integration tests, but those two easy pickings are probably the source of your down votes

Unit tests run much faster and are more focused - relying too much on integration tests to catch bugs is a massive maintenance smell, as you will waste an enormous amount of time trying to figure out the root cause of a failing test.

Thank you everyone for your feed back I am going to take this and digest it

There's a good JustForFunc episode on testing. https://www.youtube.com/watch?v=hVFEV-ieeew

I'm using table tests so that everything stays clean, and Vectra mockery for generating mocks and doing assertions.

Works great so far.

Reply on news.ycombinator.com