How to Use Go Interfaces
blog.chewxy.com
blog.chewxy.com
IMHO, if you're coding Java style interfaces up-front ("Repository", "Service", etc) you're already doing it wrong. It's likely that you're either over-engineering your project or prematurely optimizing for flexibility you will not need. You'll likely feel as though you're fighting the language to build these abstractions and layered architectures.
Go is all about minimalism. As the article says, you should be writing the concrete implementations first, then defining interfaces in the client code. The calling code should pick and choose the behaviour it needs from the implementation. As hard as it is for most seasoned Java developers, you have to silence that anxious voice inside that keeps asking "what if I need to swap this out for something else?". Go is about getting things done, not beautiful abstractions.
Interfaces are rarely rarely about interoperability even in Java. Most interfaces have one implementation. Instead, they are about abstraction, design, and so on. This article assumes that the end product is what was intended all along.
Close, with the exception of mocking interfaces for unit testing, which is very common in Java.
I am utterly sick of writing fragile scaffolding to deal with these dependencies, to the point I wish that Go provided an implicit interface for every declared type. Testing would be so much easier if I could pass in a MockFoo instead of a Foo, provided MockFoo satisfied the implicit interface.
At least you have the workaround
type PublicPart<T> = {[K in keyof T]: T[K]}; //keyof only sees public propertiesIt's awful to do in Java, but I felt fine doing it in go.
Then what you want is an abstract data type. Unlike interfaces, which can be implemented anywhere, an abstract data type is implemented in a single place. This has obvious benefits for performance (clients don't need to dynamically dispatch on operations), control (implementors don't need to worry about malicious clients supplying impostors) and precision (a single module can export several abstract types, whose implementations are mutually dependent).
Go's interfaces are pretty terrible at all of the above. Not that Java provides anything substantially better.
While Go is reinventing the languages of the 80s in the 2010s.
Guess what all nontrivial programs manipulate in the end.
> objects that don’t have such well defined invariants
... are either stateless or useless. It is impossible to define useful algorithms that manipulate objects of which you cannot say anything.
> No major language has adopted them for that reason.
The real reason is “programmers cannot reason about invariants anyway”.
(0) An ADT has a single internal representation that is known to the implementor, but hidden from users. The upside is that the ADT implementor can use his knowledge of the representation to optimize on n-ary operations, i.e., operations that act on several instances of the ADT. The downside is that ADTs cannot be extended by third parties. You would have to create a different ADT, even if satisfies the same contract.
(1) A Go interface has a list of methods, but does not proscribe any fixed internal representation. The upside is that anyone can create values of an interface type as long as they can implement the methods. The downside is that it is difficult to implement n-ary operations, because, at any point in the program, the implementor only knows the internal representation of the single value they are currently creating.
In other words, ADTs optimize for efficiency and predictability, whereas interfaces optimize for flexibility and extensibility. As a general rule, ADTs require more time than interfaces to plan ahead their design and implementation, but already implemented ADTs are easier to use correctly than already implemented interfaces.
And I don't think this is a Java vs. Go thing. Let the program, or the problem if you will, prove to you where you need abstraction.
IIRC Zed Shaw had an essay on this.
1. He uses lay (i.e. non-technical) definitions of "indirect" and "abstract" to argue that the technical definitions are somehow flawed. Words have different meanings in different contexts, but he rejects those different contexts.
2. He argues that abstract classes in Java are actually a form of indirection because the abstract modifier "creates an indirect path to the real implementation". This is after he covers that "abstract" can mean "a concept or idea not associated with any specific instance".
3. He claims that no lay definition of abstraction permits the idea of "swapping out the implementation". He fails to understand that, from the client code's point of view, "swapping out the implementation" is a direct consequence of coding to the abstraction.
4. In the words of Rich Hickey, I think Zed is conflating Simple and Easy.
I don't doubt that Zed saw something legitimately objectionable and that drove him to write the essay. But in trying to formulate his point, I think he ended up conflating ideas, relying on false premises, and built an argument that didn't have much of a logical progression.
He's essentially saying "I'm right and everybody else is wrong because I looked it up in a dictionary."
Then Start() will throw a rocket instead of starting the coffee machine. just because RocketLauncher and CoffeeMachine happen to have the same set of methods.
Sounds very much like only a theoretical gotcha.
Sure it is easy to avoid such gotchas in small code bases, handled by two or three programmers.
Scale it up to the typical sizes of enterprise projects, maintained by generation of programmers in distributed teams, with a goal of at least 10 years deployed in production and those gotchas happen all the time.
However it plays a role which programming languages I will advise our customers to use.
Currently I only advise the use of Go for those customers that need to work with Docker or K8s.
I understand the concern. I'm not just saying that, the Haskell side of me still feels it. However, the pragmatic side of me says that it's not worth giving up the convenience, power, and decoupling of not having to explicitly declare conformance to interfaces to save myself zero bugs over nearly twenty years.
Now, HN is big and I'm abundantly sure there's a dozen people who can pop up with stories of when this bit them. But it's a question of tradeoffs; literally every day I program in Go I benefit from not having to declare interface conformance. Do you face this bug literally every day? Or even every month?
("What if you did have the problem and didn't notice?" Then apparently the cost wasn't that high, so cost/benefit still tilts in the direction of doing what Go or Python does. "What if it happens as the project gets bigger than what you've worked on?" Well, the dynamically-typed scripting products I've worked on tend to be past the size event horizon for other reasons, so we could never witness this being a huge problem because the code base can't get that large without also adopting practices that would tend to eliminate this anyhow. Statically typed I haven't gotten there, though I'm thinking statistically projects tokens would tend to diverge faster than they could converge here... i.e., even if I do somehow end up with twelve Start() functions that I don't intend to be a single interface, and all manage to have the exact same function signature, if they're separated by enough in the code that they just never cross, it still isn't a problem. With Go, you still have a hard time just accidentally passing them in... they have to traverse the codebase to get to the wrong place, across statically-typed paths.)
Edit: on reflection this may actually be duck typing not structural typing.
String perform(Supplier<String> speaker) {
return speaker.get();
}
interface Broadcaster<T> {
void send(T value);
void reset();
void close();
}
void pump(Source<T> source, Broadcaster<T> broadcaster) {
// this logic is nonsense but hopefully it makes the point
T value;
while ((value = source.poll()) != null) {
try {
broadcaster.send(value);
} catch (Exception e) {
broadcaster.reset();
}
}
broadcaster.close();
}
void pump(Source<T> source, Consumer<T> send, Runnable reset, Runnable close) {
// still nonsense
T value;
while ((value = source.poll()) != null) {
try {
send.accept(value);
} catch (Exception e) {
reset.run();
}
}
close.run();
}
pump(source, broadcaster::send, broadcaster::reset, broadcaster::close);
List<String> sent = new ArrayList<>();
pump(source, sent::add, sent::clear, () -> sent.add("CLOSED"));
It's not that Go is against abstractions, really; it's just that we programmers love, absolutely love to build a system with operators and binary operators and summing binary operators and addition and numbers and real numbers and integers and complex numbers and imaginary numbers — when all we really need to do is add 2 & 2 to get 4. Go says, 'hey there, slow down, hang on a sec.' You're free to abstract if you need to — but there's a good chance you're wrong to do so.
Write the simplest thing that could possibly work. Maybe, maybe once you've written it three or thirty times, you'll have a good sense of the problem and be able to write a sane abstraction for it.
But if you're spending time fleshing out interface or package hierarchies before you even write any code: you're probably wrong.
Early and careful abstraction of absolutely every layer of your program is something you will always thank yourself for. If your classes only talk to interfaces at every single layer (no concrete classes talking to concrete classes, except for maybe a factory here or there to vend out said interfaces), your project will be much easier to test, refactor, and dynamically change.
That sort of minimial, incremental approach does not scale at all. Software engineering in the large requires, as you say, interfaces at every boundary and a near-religious understanding of how all the different components and boundaries fit together.
Fundamentally they are very different practice of software development and I think what OP talks about are those many cases where there is no scale need, never will be, yet we see this design approach that is suited for the type of projects you talk about.
Right tool for the right job after all...
If you knew what you were doing when you made those early and careful abstractions, then I agree completely (the Go standard library, after all, is a set of careful & well-reasoned abstractions). But if you made the wrong choices three years ago, because you didn't understand the problem you have today back then (either because you didn't understand the problem then, or because the problem changed), then you're going to hate yourself.
If you're attacking a well-known problem which will be the same problem you'll be attacking in a decade, then sure: early abstraction isn't premature. But how often do we do that, and how often are we exploring an unexplored problem space? At least for the projects I've worked on, each one has been terra nova.
And pretty much any language in existence. It's just healthy engineering practices.
Whether developers apply them or not has nothing to do with the language.
I think we had some good tools to solve this but either threw them away in newer languages or just forgot they were possible. Working with c recently I wanted to mock something and the language obviously couldn't help me so I came up with something along the lines of:
#ifdef TEST
#define foo mock_foo
#endif
The biggest barrier to approaches like this is the reliance on IDE's and what they can do. I've never seen an IDE that can handle rules like this, they want to compile the project and control it's structure. In c# for instance, it would be pretty easy to have a mock library with concrete alternative implementations and compile the tests individually, something like:
csc.exe MyTest.cs MyClassUnderTest.cs /reference MyMockAssembly.dll
Getting it to the point actual implementations are replaced by mocks at binary level... might be rather challenging.
(In Golang, tests mainly work like this: foo.go tests are in foo_test.go. "go test" would then run the unit tests for foo in foo_test.)
As for your test example, it doesn't work if you need to mock a library only available in binary form.
You cannot do that in Go, you need to write interfaces upfront. Or you can't unit test a controller that depends on a repository which concrete implementation relies on a database.
There is no "bytecode manipulation" here, since there is no bytecode.
Then how do you unit test your controllers in a web app? or anything that depends on a database connection without decoupling service and repository?
> Go is all about minimalism.
No, it isn't. It's your opinion about Go. Unit testing doesn't become irrelevant because you are using Go.
> you should be writing the concrete implementations first, then defining interfaces in the client code
That's not how you do test driven development via unit testing.
Just because Go is relatively new it doesn't mean software engineering best practices don't apply anymore.
That makes it actually even easier to do unit testing, because for a narrow test you don't need to implement irrelevant methods.
Unit tests test concrete implementation "units", so this is completely compatible with writing unit tests.
From the post you responded to:
> you should be writing the concrete implementations first, then defining interfaces in the client code.
When you need to take an argument that you can swap out for another type, that's when you define your interface. Not earlier.
> Unit testing doesn't become irrelevant because you are using Go.
Unit testing doesn't become irrelevant because you don't architecture astronaut from the start. When you need to test, define an interface. But only the interface that you need when you need it.
> Just because Go is relatively new it doesn't mean software engineering best practices don't apply anymore.
By providing the client the ability to define the interface it removes the need to pretend you can plan for all circumstances upfront.
Instead, at the moment you need to be able to mock things, you make an interface and mock them.
That's not to say that you should never provide interfaces in a library, just that it's no longer the only option available.
I'm not sure why you think that unit testing becomes harder with this approach, because to me it seems far easier.
I admit it, I make Repository and Service quite frequently. With that said, I also have data types shared among implementations. Normally the data types are defined within the parent packages to the implementations, the same packages that hold the Service definition.
I actually agree with your assertion that Service is bad form, and think I'll change it. I still have the issue though where a shared data type is commonly used among many implementations. Where do you think that will live? Perhaps in the same location, just without a formal Service interface?
> “Accept interfaces, return structs”
Real example from personal (professional) life against that: Amazon AWS SDK defined S3 as a struct. That makes it bloody hard to mock during unit testing. I eventually ended up defining my own “s3iface” interface with corresponding mock to combat this, but now I have to repeat this for every program that uses it. Ok, I’ll separate it into its own package and include that everywhere. Wait, what was the point of the AWS SDK not doing this directly, again? Oh, look, they eventually ended up doing just that: providing an interface version of their S3 struct:
https://godoc.org/github.com/aws/aws-sdk-go/service/s3/s3ifa...
> Package s3iface provides an interface to enable mocking the Amazon Simple Storage Service service client for testing your code.
With the exact same name, to boot.
Please consider providing and returning interfaces. Even if your struct is the only type implementing it.
Interfaces are light-weight, and it's okay to define an interface that is just the subset of functionality your code needs.
This is exactly what the GP is saying.
I realize that this goes against the Go ethos of "don't think about the future; think about the now". Having said that, I don't understand why returning interfaces is seen as premature design while accepting interfaces is seen as reasonable and expected. Surely both can be examples of premature abstraction.
I'm certainly not saying that you should always return interfaces, or that returning interfaces will prevent all future refactoring pain. But working with interfaces instead of concrete types is more likely to allow your API to remain stable even as the implementation evolves.
Can you share why? Not trying to say you are wrong, but I am guessing that part of this stems from some limitations in the way interfaces work in Go, and this could make a good user experience report.
I am not sure if I agree with
> Defining an interface upfront is usually a code smell for overengineering.
So far I had less problems with overusing interfaces than with not using them enough. So when you write tests for a package, you define the general interface to the package in terms of types and methods. The remaining challenge is to break that general definition into smaller pieces as one big interface for a package is almost certainly a bad idea.
However, in general I follow the authors verdict to not just build 'Java-style' interfaces, as that completely misses the point of interfaces in Go.
So in fact, I don't like to have circular dependencies in Go either and try to avoid them, so I agree with you on that. But at the same time I like very small and simple packages and sometimes you have to choose:
1. do you build a larger package with everything inside
2. do you build multiple smaller packages with dependencies on each other
As I said, in general, I favor the smaller packages and that can lead to circular dependencies, if you are not using interfaces properly.
But I still think that you have multiple packages and each one is calling all the other ones, then it doesn't make much sense to have multiple packages at all.
I prefer smaller packages too, but currently I'm trying to avoid overmodularizing, specially when a package is too connected to some others and is not going to be used elsewhere.
Furthermore, taken at face value, Postel’s law was a phenomenal failure in principle and in practice (see e.g. [1]). In hindsight it’s probably a bad design principle.
[1] https://tools.ietf.org/html/draft-thomson-postel-was-wrong-0...
People seem to really click with the net/http design, and that does seem well done, but I imagine the standard library itself is a bit unevenly architected.
And while NewAES might predate this advice, NewGCM (which returns an opaque crypto.AEAD) does not.
> Of course there are subtleties that I am eliding. For example, context-sensitive code would benefit very much from taking interfaces and returning interfaces
Or to steal from Haskell's lexicon: when you do something in a monad.
Serious answer: Not really. Monads are more abstract than just contexts. Monads are commonly used to represent context and contextual computation, but they can represent other things too.
What's the similarity between List, State, and IO? I'd argue that there is little similarity apart from the fact that we can define the monadic operations for all three. And, having defined those operations, we can then use more complex operations like `sequence`.
Monadic types sometimes represent data structures, sometimes represent contexts or, in the case of IO, sometimes represent nothing really at all. IO is arguably just a marker that imposes a linear sequence on what would otherwise be unsequenced computation.
In short: don't spend too much time trying to understand monads as a whole. Rather, try to understand why specific types, like List and State, happen to be monadic.
Other than that interface definition should be done at the call site, not the implementation site. This is true in Java also. Golangs structural typing doesn’t change that, it just removes boilerplate in the most trivial cases.
I guess you wrote it as a continuation of the opinion from the article that interfaces should only be used as input.
From my perspective, I was still considering all other cases also. For other cases, it's precisely because of Go's structural typing that such a thing can even be considered. Java can't implement an interface for a type it doesn't control. You could create a wrapper class, but not exhaustively wrap all present and future implementations.
To decouple the use from the implementation we generally are either adapting the implementation or narrowing it. In the latter case, structural typing saves you boilerplate, in the former it likely doesn't.
The OOP consultant set calls the the concept we are dancing around Dependency Inversion. In Java it very routinely results in wrapper classes. In the most basic cases those classes are pass through boilerplate, but those are fairly rare in practice. Golang dismisses with those.
In more cases in practice, you are actually defining the interface that matches the call site demands and modeling different implementations to that. Golang's structural typing doesn't help much with that.
I tried (not terribly successfully) to write up some of this in the DIP section of this http://kc.my-junk.info/di-ioc-dip it uses scala for the code samples, but it should be fairly easy to see that golang doesn't remove much of the wrapper code if I'd used that.
> The most obvious of which is that it has only one type that fulfills the interface with no obvious means of extension. (sic)
Seems obvious to me.
type Cat struct{}
func (a Cat) Speaks() string { return "meow" }
Does this not work?
> Functions typically take concrete types instead of interface types.
Ok...? This seems like some arbitrary rule that the creators of Go claimed was a good idea, I guess. The author has offered no justification for it. In general I guess I fail to see most of the proclaimed benefits of Go, every time I read articles like this I come away favoring something like Java for anything in the application layer. I can see preferring Go to C for low-level and systems programming, but I shudder to think of what a mess the large Go codebases must look like.
> Does this not work?
It works
However, this is what I have recently found quite a lot. An interface is defined:
type Foo interface {
Method1()
unexportedMethod()
}
This is typical when developers come from Java and they do `abstract class Fooer` and immediately do `class Foo implements Fooer` within the same package. This is an anti pattern in Go.
> This seems like some arbitrary rule that the creators of Go claimed was a good idea, I guess
No. I think I was unclear. I specifically mention that it's not a great idea. It's a code smell. Instead, interfaces should be defined at point-of-use.
You still have not said WHY it's not a great idea, or what the benefits of defining the interfaces at point-of-use are.
Imagine you have two packages: A and B. A defines interface Fooer and struct FooThing. Package B only requires anything that can call Foo(). But because the interface is defined at A, now B is forever tied to A. When you change something in A, you have to worry if it changes something in B as well.
If instead the interface Fooer is defined in B, package A can change as much as it likes without B needing to change.
It's one of the reason why I think Go seems to draw more people from dynamic languages than static because the static folks have a tendency to shoehorn some old habits into it.
They could match with all the interfaces defined at the standard library (io.Writer etc.) plus the interfaces defined in that same package.
Well, I guess it's better than nothing.
Besides the fact that there is zero justification from this claim, the simple fact that Go think it's important to distinguish the two is another illustration of late 90s programming language design.
There is really no good reason to separate these concepts today, it only causes confusion and fragments the code base in arbitrary chunks that accumulate smelly code over the years.
But I'm happy to hear a good justification of why it's important to have both structs and interfaces, if there is one.
Performance is a good reason. If you are familiar with Java or C#, Go interfaces can be seen from one point of view as boxing (i.e. making a primitive value an object). Think what that would require: vtables for function dispatch.
A bunch of inlining opportunities cannot happen with that (well, it can, if you allow for name mangling during your compile phase, but that's very gross). JIT is the second fix to that, but with JIT, you're bringing in a lot of unpredictability - useful in some contexts, not so in others.
Perhaps you are fine with the "Everything Is An Object/Function" paradigm, but I'm not entirely fine with that.
If it has some implementations and some without, it's an interface with default implementations.
See how Kotlin does it, it's really nice and consistent, and you never need to agonize over the decision of picking a class or a struct.
The clearest benefit is the decoupling of dependencies which leads to more testable, extensible and maintainable code. That any struct can simply implement the desired behaviour to implicitly satisfy externally defined interfaces is an extremely powerful notion.
Of all the things to complain about in Go, interfaces seem like a strange bone to pick.
func f1(r io.Reader) {}
func f2(r interface{ Read([]byte) (int, error) }) {}
func f3(r interface{})
One practical usage:
li, _ := net.Listen("tcp", ":9000")
for {
conn, err := li.Accept()
if err != nil {
if err, ok := err.(interface {
Temporary() bool
}); ok && err.Temporary() {
time.Sleep(time.Second)
continue
} else {
panic(err)
}
}
}
switch err := err.(type) {
case nil:
case interface{ Temporary() bool }:
if err.Temporary() {
time.Sleep(time.Second)
continue
}
panic(err)
default:
panic(err)
}
All that to say, creating duplicate interfaces is perfectly fine.
"codes?"
Plural form of this type of code is "code."
Stopped reading there. This is one thing I just can't get past when I see it.
Yes, I realize that I am not a superhero-level altruist.