Avoiding Reflection (And Such) in Go
jerf.org
jerf.org
Further, it makes Godoc more confusing, since now I have to search around to figure out what each of these custom types is.
This rule goes way beyond "tiny types"; for instance you will drive yourself insane in the web world if you don't do this, with double-decodings and double-encodings quite challenging to prevent with any other policy. [1]
"Further, it makes Godoc more confusing, since now I have to search around to figure out what each of these custom types is."
Absolutely yes. In any language where this is possible and easy, I do wish the auto-doc systems either automatically, or with some metadata (preferably a label on the type somehow), would make it clear when something is just a one-layer wrapper around a simple type in all the places where it appears in the docs, without having to click through. I consider it worth the cost, but this is definitely an entry on the cost column that is accidental, not essential. This is especially true when the base type is numeric, which means that Go will let you pass in a literal number in the source code, but that is not always your first expectation.
[1]: In a statically-typed language you can also play games with having the type label what "layer" of decoding the value is currently in, which works but gets more tedious. This is theoretically possible in a dynamic language, but goes against the grain, since generally people program with just 'strings' in those languages.
Some of the fancy IDEs help with this problem, but not everyone likes using them so you can't count on that as a solution.
You have some good points about tiny types, but if you read through the code in the Go standard library, you won't find this style applied in very many cases. Typically, only if one wishes to attach a method to a built-in.
As another poster rightly pointed out, it complicates the documentation quite a bit, and I think a lot of Go programmers would consider it pointless abstraction.
// NewBox creates a new Box. It returns an error if length
// or width are less than or equal to zero.
func NewBox(length, width float64) (Box, error) {
...
}
Go puts a big emphasis on documentation, and this often compensates for lack of power in its type system.
This could be reworded to:
> Any language X that puts a big emphasis on documentation compensates for lack of power in its Y.
I disagree this is true for those languages including Go.
Simplicity is not a weakness or failing of a programming language, it is something that every language designer should strive for. To imply otherwise is to say (non-essential) complexity is a good thing.
It's a small, simple, language that is terrifically pragmatic. Write code that's clear, self-documenting, and works. Then go back and fix up performance, convention, and style, as needed.
I wonder how many thousands of hours has been spent talking back and forth over an ultimately trivial part of Go.
......
Solid articlie though. As someone who writes a lot of Go, is sick of hearing about Go, I enjoyed it.
Go's (really Plan 9 C's) composition support does "intermingle" methods, which has pretty much the same effect as intermingling fields. To see this, note that Go needs rules about who wins if methods for an outer object and an inner object have the same name (and they have to consider diamonds, which you can have with composition!)
> So, "Go doesn't have generics" is only half true.
I know this is going to be me annoying about terminology, but I really disagree. :) Go has polymorphism. It doesn't have generics, at all. "Generics" has a specific meaning: the ability to take types as parameters.
> It can be profitably argued that the primary virtue of "generics" is generic algorithms rather than generic types.
I don't agree. Generic types are necessary, because you often can't express generic algorithms naturally without generic types. Just as an example of where I needed generics in my work most recently, I wrote a polygon clipper over the past couple of weeks that has special fast paths when it clips rectangles and falls back to Sutherland-Hodgman clipping for complex polygons. This was natural to express by making the clipping result generic over the type of polygon (rect or edge list). I could maybe have done it in Go by using the "container-as-interface" trick that the sort package uses, but that would introduce a lot of boilerplate.
There's also the issue that all these interfaces make it awfully hard for the optimizer to produce as good code as it would if you actually had generics. Unless you inline everything away (in which case SROA and constant propagation can mostly get you there), you'd need a nontrivial interprocedural analysis. Performance issues around generic programming tend to have outsized effects, because functions that tend to be generic are precisely those functions that tend to be called a lot (because they're generic!) and shoot to the top of the profile.
> As a side note, if Go ever does grow full generics, I expect it to do so by extending the half it has, rather than adding a new concept of generics on the side. It's easy to imagine a generic BalancedBinaryTree that requires its nodes to conform to a LessThan interface.
This I agree on entirely. Note that this is pretty much exactly what Swift did :)
> If you have a constrained set of messages, for instance, you can use the type system to more carefully declare them as coming from a particular set of types, not just interface{}.
But you lose half of the point of sum types, in that the compiler can check exhaustiveness for you. It's worse, because since you're doing a Go type switch, you can match against cases that aren't even in the sum type and the compiler won't complain! It's also an awful lot of boilerplate to create a half-approximation-of-a-sum-type, meaning that the ergonomics favor people not using it. interface{} is just plain easier to use compared to the sum type pattern, which is why it gets so much use.
> Go still manages to cover so much with its meager feature set. There's a lesson here for language nerds. I don't fully know what it is yet, but I'm working on it.
I think the lesson is "people will figure out design patterns to approximate things that your language doesn't support". This is no surprise: people write OO systems in the C preprocessor. But I think it would be a mistake to conclude "omit useful features from the language if there's any way people can possibly approximate them". This was the approach that Java famously took, and people rightfully soured on the extreme emphasis on "design patterns". And what you've described in this post is nothing more than a set of design patterns. Design patterns have their place and are very useful, but they're also the sincerest form of feature request.
Just for clarity, what I mean is that when those methods are called, the get the specialized type, as opposed to multiple inheritance where the two base classes can end up calling into each other in a mutually-derived child because the methods still receive an object of the full derived type. You certainly can end up with conflicting method names. I have to admit I don't dock Go much on that, because as we all know, there are two hard problems in computer science, caching and naming things.
"I don't agree. Generic types are necessary, because you often can't express generic algorithms naturally without generic types."
I'm pretty sure I basically said that latter part.
"...polygon clipper..."
I didn't repeat it in this post, but in other places I actively volunteer my belief that anything that involves "doing lots of math" (in the full sense of the term, rather than "a lot of arithmetic") is a bad place to use Go. But I'd observe that every time someone reaches for a reason why they really badly needed generics, it always seems to involve math. I expect this means something.
(It's OK, you don't need to go searching for one that doesn't. I fully believe they exist, and I occasionally encounter them.)
"I think the lesson is "people will figure out design patterns to approximate things that your language doesn't support"... And what you've described in this post is nothing more than a set of design patterns."
I'm not sure that's supported by the text very well. The only thing in there that probably qualifies as a "design pattern" is the sum type approximation; everything else is too small. The only "design pattern" I find myself frequently reusing in Go is external API + "sum type" protocol + goroutine server receiving that protocol. (The boilerplate in that case is the external API implementation, and the use of a channel to return a value; the message types and the server loop themselves are generally not redundant.)
I cut this off the end, but maybe I shouldn't. More what I'm going for here is that I've got a simple language that does a good job with X% of the problems of the world, and over there, I've got a language with a laundry list of complicated features that are much harder to understand and work with, and it goes a good job with Y% of the problems of the world, and I think there's a lot of people who master one of those hard languages and come away thinking X is roughly 5% and Y is roughly 100%. But especially from an engineering-in-a-large-team perspective, it's worth being reminded that that's a cynical estimate, not a reasonable one. It varies from place to place, of course, but X can easily be 80% and Y merely 95%, and might want to try to be sure that you're only paying the price for Y when you're in that other 15%.
I'm curious whether it's possible to build a better language that covers more of the space between Go & Rust, but with a more Go-like cognitive footprint, and I'd like to poke interested people to consider if there might be an answer to that. Again, especially from a programming-in-a-large-group perspective, I still don't really love any of my options. I realize that from your POV (no sarcasm) Go seems like a terrible downgrade, but from my POV it has been an immense upgrade from a world of dynamic scripting languages, in terms of what I can guarantee and provide in my APIs without writing in a language that virtually nobody understands. There's no way I could put Rust or Haskell or anything else that strong and expensive into where I'm putting Go. Go's in a sweet spot that happens to be poorly occupied right now, and I would love nothing more than for someone to improve on it. This is especially true because other than perhaps getting real generics added on later (which I still think is at least conceivable) there's a lot of other places where Go is already very near its own local optima; it's not going to improve much, I think. It will be pretty much what it is.
It's no secret at this point that I disagree with lots of Go's language design decisions, but it's undeniably an excellent tool, whatever its strengths or weaknesses as a language are (hat tip to tptacek for this phrase). The Go team and community did a great job of making it very practical.
Either way, I don't think this example proves the necessity of generics. Some algorithms are certainly more difficult to express in Go (graph data structures/algorithms in particular in my personal experience).
Well, sure. But the title of the article is "Avoiding Reflection in Go". :)
The provided tooling provides templating, which isn't dissimilar to where many other languages stop in their generics journey. The interface does leave a lot to be desired, granted.
func max(a, b T) T {
if a > b {
return a
}
return b
}
$ gofmt -r 'T -> int' -w max.goWhat distinguishes templates from macros is that templates deeply interact with and are instiantiated at the time of typechecking. Macros, on the other hand, are expanded at parse time (or, in more sophisticated implementations, at name resolution time).
# Common pattern in other languages
max<int>(1, 2)
# Similar pattern in Go
#go:generate gofmt -r 'T -> int' -w max.go
max(1, 2)
Although you do have me curious how you'd implement max in C in this type-safe way using standard tooling:
#define max(a, b) (a > b ? a : b)
#define defmax(name, type) type name(type a, type b) { return a > b ? a : b; }
> Which is a shame because C macros just aren't good enough.
No disagreement here. However, that still perfectly satisfies what may people claim they want in generics, especially those who most frequently trumpet that Go is lacking them. If we took that exact macro and added a little hypothetical syntactical sugar:
template<type> type max(type a, type b) {
return a > b ? a : b;
}
max<int>(1, 2);
template<type> max(...) into #define defmax(type) ...
max<int>(...) into defmax(int); max(...)
Well, sure. Hand-written assembly is also identical to compiled code, but we still have compilers :)
> All you are gaining is prettier looking code, which shouldn't be discounted, but generics are about more than that.
Yes.
> Yet this example would be quite satisfactory to many regardless; curiously even you seemed excited about it.
I don't think macros are a satisfactory replacement for generics, no. I don't know how I seemed "excited" about it.
I mean when you convert the template<type> statement to the #define statement with the hypothetical preprocessor, it would be the same as if you had written the same code using #defines originally. They are functionally interchangable. If this satisfies generics in your mind (I don't think this describes you, for what it is worth), then standard C macros are more than sufficient to cover all the cases you would want, albeit less pleasant to type.
> I don't think macros are a satisfactory replacement for generics, no. I don't know how I seemed "excited" about it.
Consider your previous post misinterpreted then. It read to me like you thought the hypothetical code was theoretically different and solved problems with the previously provided macro, even though it was the exact same macro in both cases. The only difference was the slightly different syntax. Which, I might add, anyone can fairly easily add something similar to their Go code if they really wanted to.
No, it doesn't. Not even close.
Even something like generic List<T> and sort<T>(List<T>) (separately compiled) will fall down because that simplistic scheme doesn't give each generic type and function a different name, and there's no way for sort<T> to look up whatever name List<T> got instantiated into.
There are also all sorts of hairy issues you will get into when you have name resolution and separate compilation: how do you make sure the macro got instantiated into the same lexical environment that it got declared in? This matters a lot when you have package A exporting a bunch of generic types and functions for package B to use, and it's going to be B doing the expansion using B's types. Even if you set up things just right to resolve the names in the proper lexical scope, how can B's expansion of A's templates use private symbols from A?
Finally, you have the dreaded C++ template error message problems, except they're going to be even worse with this scheme because the compiler is going to complain about code post-textual-replacement, with no way to interpret the chain of events that led to the error. At least clang and GCC have lots of semantic information that they can use, since templates are expanded during semantic analysis.