Generics in Go with Ian Lance Taylor (2019) | Hacker News Reader

Generics in Go with Ian Lance Taylor (2019)

changelog.com

92 pointspjmlp98 comments

I just don't want to hear about Golang generics any more, until they are in a usable beta. It's been years, and while I'm confident that there are some maintainers who really for real want to add it... it's hard to shake the feeling that the talk is going to go around in circles, hoping devs give up and forget about it.

It's like sending a project into an environmental review. It sounds constructive, until you realize it's a nice way of black-holeing the project.

I really apologize if I'm being unfair to the (team?) working on this, but... it's been years. Generics have been around for decades. It's important to get it right, but Golang is by design _not that complicated_. There really doesn't need to be wild new R&D to figure out how to pack generics into a programming language.

</salt> rant over. For now.

Edit: And seriously, it's 100% fine if the answer is "we thought about it, and decided not to do it." It's just frustrating being strung along with constant design proposals, iterating on syntax, and then... back to more design proposals. Just make a choice (or a veto), and go with it.

At this point, I tend to agree. The original creators are quite fine with "Me but not for Thee" so generics simply aren't in scope. The only way generics will come in scope is if Go starts losing enough popularity that the creators need to do something. That isn't likely to happen for a while.

To the users, if you want generics, WHY DID YOU CHOOSE GO? There are plenty of other options nowadays.

Ken Thompson and Rob Pike are mostly our not at all involved with Go anymore. They are 2/3 of the original creators. It’s different people making the decisions now

>Ken Thompson and Rob Pike are mostly our not at all involved with Go anymore.

I'm fine with Ken, but if Rob is not involved, things might progress faster. I have a big impression he was the impediment in most cases...

>To the users, if you want generics, WHY DID YOU CHOOSE GO? There are plenty of other options nowadays.

But I didn't choose Go and I do want generics.

>To the users, if you want generics, WHY DID YOU CHOOSE GO?

My employer chose it. So there's that.

Go is an opinionated language, IMO its developers need to make a decision about this. Either get on and add generics (as in one of the existing proposals), or (given the language’s success in its current form) explicitly say “no generics, the fundamentals of the language are frozen”.

I suspect the problem is that Generics bring you face to face with the Liskov Substitution Principle. For most of us it's just a good idea, and then you try to do generics and all of a sudden it begins to become the law.

So you try to layer it onto a language like Java and you discover that an ArrayList of Integer is not interchangeable with an ArrayList of Number because you can stick a Double in one of them and blow up everyone expecting only whole numbers.

I've long suspected that you either have to put Generics in at the beginning or prepare for decades of complaints about your workarounds. And getting stuck in committee for a very, very, very long time.

> So [...] you discover that an ArrayList of Integer is not interchangeable with an ArrayList of Number because you can stick a Double in one of them and blow up everyone expecting only whole numbers.

Is this really an issue with Java's type system though? If you get an ArrayList<Number> and you assume that it contains only whole numbers, that's on you.

FWIW, my personal opinion is that erasure is better than templated generics for most use cases. Even when it comes to performance: a good JIT should be able to specialize generic code that end up on a hot path. Erasure does cause some issues with reflection and value types, but I think that it provides a better foundation to start from when designing a type system.

Then again, I am quite a fan of Rust's generics and dyn/impl {Trait}. Might be worth doing something similar where you have two separate generic systems: one for boxed values, and one for unboxed values.

> Is this really an issue with Java's type system though? If you get an ArrayList<Number> and you assume that it contains only whole numbers, that's on you.

The point is that you have an List<Integer> are you allowed to call a function that takes an List<Number>? The answer is effectively "yes it should be allowed" if its not mutating the List (all Integers can be safely read as Numbers) but no if the List is mutable (you can't handle the fact that it might insert a non-Integer number into the Integer-only List).

This is less LSP and more the co/contravariance of container types, which is its own special hell.

The hell stems from inheritance, which is just syntactic sugar over composition. And a very unhealthy kind of sugar as that: problems with co- and contravariance, problems with copying, the need for redundant type declarations. And inheritance doesn't guarantee autofulfillment of the Liskov principle, either. One might even wonder why inheritance is needed at all in a language. Go and Rust are certainly fine without it.

One of the humblest smart guys I worked with years ago would try to convince me over lunch that the Next Big Language would have syntactic sugar to make composition as easy as inheritance. I think in part because I had intimated that I was trying to create a language (narrator: he didn't. It went nowhere)

There are a couple languages that have added something like this in the interim, but nobody seems to have made it an idiomatic thing. Which is a shame, because I still think I might like to work in the world he dreamed up.

You get a little more composition in languages like NodeJS for the very pragmatic reason that inheritance, at least historically, was not seamless and hence had a high degree of friction.

Ah, just like VB did with COM....

With the possible exception of a couple of convenience methods, you can model collections (independently) as a source and sink for elements and avoid having to rely on variance at all. But we treat them as a storehouse of shared state... which ought to be our first warning that things are about to go sideways.

Lots of people get confused by variance. They shouldn’t, but insisting in what people should or shouldn’t do rather than working with how they are is a sure route to frustration.

It isn't a problem if you design an API by following the rules (PECS). Use wildcards for method parameters when this is the desired behavior. It is not hard to do things right.

https://howtodoinjava.com/java/generics/java-generics-what-i...

void doThings(List<? extends Number> values){ ... }

This is catered for.

> because you can stick a Double in one of them and blow up everyone expecting only whole numbers.

Generics in Java are invariant, so what you're describing will not work.

Now imagine being the dev who keeps making proposals and working diligently at the problem only to have countless ideas and proposals shot down for one reason or another. Ian Lance Taylor is persistent if nothing else and I'm hoping his hard work pays off with this proposal.

This is why we’re going round in circles. There’s been significant effort invested in generics for Go, so the team aren’t willing to veto the idea. But they’re not willing to accept a proposal either.

The team is actively working on it. Why would they invest work in something they don't want.

Working on it _internally_, in depths of google sea. There is not much to follow that progress other than following small bubbles from that submarine raising to the surface here and there... (imho, ymmv)

Not everything has to be followed at every step. When they have a relevant update on the draft design they publish it, gather feedback, improve, and so on.

People want to follow it closely. And after that fiasco with "new" error comparison handling proposal I cannot blame them.

Ian's talk about generics at gophercon 2019: https://www.youtube.com/watch?v=WzgLqE-3IhY

The proposal seems so simple, I want to use it already! It would be a nice idea to have a translator from generic go to stable go, so that users of go can start using these generics.

Edit: an implementation was linked in the talk as a changelist to try it out: https://go-review.googlesource.com/c/go/+/187317/

Anyone know why they didn't go with angle brackets? People familiar with C++ and Java would find it familiar. I wonder if its too unpythonic, or makes the parsing too hard.

e.g. Instead of

    func Reverse (type  Element) (s []Element) {

why not do:

    func Reverse<Element> (s []Element) {

Ian mentions it would make the parser contex aware, which isn't needed currently.

That at least isn't true in terms of the Chomsky grammar hierarchy. You can have both angle brackets operators and delimiters in a CFG (context-free grammar). It might be a limitation of their parsing tech (e.g. LL(1) would have that issue, probably LR too).

No problem for general CFG parsers though (if you can write a BNF grammar for it, that's the CFG grammar for it).

Most likely, just repeating his answer on the podcast.

Parenthesis are not binary operators so the parser doesn't need to know whether LHS and RHS are identifiers or expressions.

This is the same reason why D uses a bang (!) for template instantiation (as well as this exact syntax for declaration) - I think D is the benchmark for generics in 2020, especially alias arguments, of the C++ like languages anyway (so no Haskell!)

I also think the contract system this design and C++ have gone for is overly complicated compared to the highly extensible and extremely simple system D uses (just write if(ContractCondition!Type) between the template declaration and it's body

    l := list<int>(lst)

    l := x < y

When it reads the "<" sign, the parser doesn't know whether it is in the first situation (an instantiation of a generic type) or the second one (a comparison). It makes parsing harder.

It's even worse when the actual type is also a generic one, as in list<list<int>>. That's the reason why you had to write `list<list<int> >` (note the space) rather than `list<list<int>>` in C++ until C++11.

> ... using angle brackets, but we couldn’t figure out how to make that work in Go… Because Go has the ability that you can parse the syntax without knowing the types of the names ...

This is not the first time I heard that keeping the parser simple is more important than others in Go design from Go core team. This might be good sometimes, but I think this will hurt user experience more often.

You could argue that the trade offs are worth it. Compilers can be faster, auto format works, autocomplete, code snippets, merging etc.

Maybe it's a bit off topic, but I wonder how much the Kubernetes code base influences this kind of demand to add features to the language.

Why Kubernetes in particular?

It's one of the largest open source ecosystems written in golang by lines of code, it's also a major priority for Google.

Because it’s huge, very abstract (pluggable components are a first class concept), very publicly-known (it’s name carries weight), and has numerous numerous active contributors, whose membership has evolved continuously over time. It makes use of a lot of magic to get over a lack of genetics, and has innovated in that case.

There aren’t a lot of other projects with that profile in Go. etcd and raft are similar, but are maybe an order of magnitude less “big”.

because the k8s project is full of generated code, which really hurts the readability.

Quite a bit I think. Go module system did cover many use cases which Kubernetes cared but other systems may have not.

Kubernetes wasn’t an influencer. In fact go modules require semver and kubernetes doesn’t use semver. There are little things like that

One concern I have about the shape of the generics proposal as it stands is that it makes it hard to use arrays of types inside of generic data structures. Block-based data structures are a staple of efficient programming. Having a generic B-Tree-backed sorted map would be wonderful. Also wonderful would be a library implementing a dequeue backed by a linked list of ring-buffers.

Maybe I'm missing something in the proposal but it seems like it's awkward to build these in the current proposal with contracts as type lists and methods. One option is to implement a `slice()` method on your array types and then interact with them as such.

```

interface Array(A, T) { A slice() []T }

type Dequeue(type A, T Array) struct { ... }

```

Then you'd do something like:

```

type intArray8 [8]int

func (ia intArray8) slice() []int { return ia[:] }

func newIntDequeue() *Dequeue(intArray, int) { ... }

```

But there's something that feels dirty about that.

Golang is a pretty much messed up language made by short-sighted and very, very, very stubborn designers. If you don't believe me, just look for instance at all the hacky code generation tools made by the Kubernetes guys. But even if generics are there, the language itself is still very lacking to an irritating level (e.g. no option types aka nil access, no enforcement of error checking aka no result types, no enums, no conditional compilation, no iterators, no immutables, etc...)

EDIT: HN upvote system has nothing to do with right or wrong or constructing a good discussion, it's based on fanboyism and political correctness. Really sad.

Go has been amazing here at Stream. No way we would be able to power chat and feeds for 500 million end users with such as small team without Go. C++ would be a valid option in terms of raw performance, but the development overhead is just too large. Go hits a beautiful sweet spot between performance and developer productivity.

I am not sure how you refute anything the above poster said. They did not mention performance once (cough Rust cough) and that seems to be most of your point. You did not even say why go is so good for developer productivity... the poster made many points that could be used to argue that go does not promote developer productivity.

He's an employer, and like I said below it's not surprise that all employers and managers who rarely write any code defend Golang whenever the debate shows up. The language features barely have anything to do with the language itself (easy to pickup -> bigger pool of developers and lower average paychecks/dev, acceptable performance and memory usage and fast compilation times -> better AWS bills compared to Ruby/Python/Java, abundance of support for many cloud and distrubted stuff (e.g. k8s go client, envoy, cloud vendor sdks, etc...)

The data I’ve seen actually suggests Go developers are paid on average better than most other stacks: https://insights.stackoverflow.com/survey/2019#top-paying-te....

I am not sure whether these statistic reflect anything on the real world, but even if true this is due to the nature of applications in which Go are used e.g. distributed systems, cloud native, etc... and its concentration in the bay area maybe

> the poster made many points that could be used to argue that go does not promote developer productivity.

I think go promotes maintainability rather than developer productivity. IMO, go authors favored ease of code reading to the expense of code writers. That being said...

> no option types aka nil access, no enforcement of error checking aka no result types, no enums, no conditional compilation, no iterators, no immutables, etc...

No enforcement of error is not true, you have to explicitely ignore an error if you want to. You have almost the same problem with rust, where you can unwrap things that can fail, thus explicitely ignoring potential errors (granted, the program will then panic).

No conditional compilation is not true either, you can, eg, put at the top of your file build tags:

   // +build !linux

This file won't be compiled on linux (you supposedly have another version of that file for linux systems).

No immutables: can be a problem, const is very limited indeed in go.

Nil access, yes, although contrarily to C/C++, you can't dereference nil without having the program panic. AFAICT I never had them happen in production, when I have a panic it's because of an off-by-one error in a slice, usually (but then I'd have the same problem with `safe` languages like rust).

I'd be glad to have enums. Go's workaround is safer than C's enums, but not by much.

> I think go promotes maintainability rather than developer productivity.

It seems to promote verbosity just for the sake of so called "simplicity". It's simple (almost dumb) at the language level, which just pushes complexity elsewhere.

This is just a statement that I find some people repeat without any substantiation whatsoever.

> you have to explicitely ignore an error if you want to

    err := foo()
    ...
    err = bar()
    if err != nil { panic(err) }

The first error was unintentionally discarded. I've seen this happen in actual code bases.

> It seems to promote verbosity just for the sake of so called "simplicity".

I don't think simplicity is the goal, I think simplicity is a mean to an end, and that end is readability. I remember reading some C++ code I didn't write, and I couldn't understand where the bug was coming from. Turns out the developer had redefined the () operator (or something like that) so that it behaved quite the same as expected, except in a few corner cases.

That's the kind of bug hunt go preserves you from, but verbosity is the price you have to pay.

> The first error was unintentionally discarded. I've seen this happen in actual code bases.

You're right, and AFAIK linters don't catch these.

I would have said so as long as I am the employer and not the one who actually work with the language.

Curious if there’s a reason Java/Kotlin wouldn’t have worked? There are a number of fast, statically typed languages with much less developer overhead than C++.

Do any of them have comprehensive standard libraries, and compile to small native binaries? Those features, along with performance and support for parallelism, are why I chose Go (for command-line apps).

Small native binaries don't seem like much of an advantage when writing servers. For command-line apps sure, but it sounds like OP was addressing someone who was writing a server application.

> Small native binaries don't seem like much of an advantage when writing servers.

However when deploying server binaries it's a substancial advantage. End users like them. Ops like them.

End users happily chug along with 300MB Facebook Messenger on their phones.

On server I am deploying 250MB TensorFlow on a _free_ cloud tier.

Not sure what are you talking about.

I also bet Java or .NET would be speedier than Go after warmup.

Small native binaries don't have to warm up when you start / restart a server.

Sure they do, they have to warm up the cache and OS data structures, whence why Windows has a pre-fetch service.

The last time I gave these reasons for choosing Go, that post got downvoted as well. But, still no-one can suggest a more appropriate language to use.

I don’t even particularly like Go. It just seems to be the best solution for command-line apps in 2020.

Because plenty of managed languages do support AOT compilation to native code, including Java.

With GraalVM you can compile JVM languages to small executables with limited memory consumption. It's fairly new technology but people are using it in production.

Yes, since around 2000's when willing to pay for third party SDKs.

Nowadays OpenJDK, OpenJ9 and GraalVM offer AOT for those that aren't willing to pay for such SDKs.

As for small, Go binaries aren't necessarily that small.

Java can be as fast as Go if you are careful. Unfortunately idiomatic Java isn't terribly fast because of the pointer chasing behaviour in its collections. The only way to avoid the issue is to work primarily with arrays of basic numeric types. If you sort a large vector<Point3d> in C++ it will be an order of magnitude faster than sorting a similar looking ArrayList<Point3d> on Java because the C++ version stores and allocates these tiny objects in a contiguous memory block.

Java is also going to take a ton more memory.

Java doesn't need the trick to allocate useless GB to force the GC to behave, like Twich and others have reported about.

I also bet that Java value types will be available in the language earlier than any Go compiler with generics support.

https://jdk.java.net/valhalla/

FWIW, I’m a happy Go user who agrees with the direction of your misgivings, if not completely the magnitude.

There is a lot of hacky code in the code generation space, which is why I think the Go team continues to chase generics rather than give it up completely.

A lack of optional types is IMO a wart, that’s tapered over with nil and reflection, but would have benefitted from being a first class piece of the language. I like to imagine this finds its way into Go 2, but I’m not very hopeful.

The rest of your items I view as very nice things that I don’t particularly miss. They’re quirks that are reflective of a minimalist design. At least, that’s the excuse I make.

Result types are interesting and valuable, but add a layer of complexity to a program’s legibility that is understandably desirable to omit, especially when the idiom of returning an error type is so culturally ingrained.

I miss immutables and “modern” iterators, but I can personally live without them. Conditional compilation is kind of antithetical to Go’s design and I don’t miss it too much.

I think most Gophers who have been around the language a while would agree with most of what we’ve said, at least the ones I know.

Go's strength is not the language.

But there are so much more to an ecosystem than the language itself. People pick Go, despite its language shortcomings. That should be telling to a lot of other ecosystems out there.

Also, the niche it carved out has no other language that fits equally well, I think.

Can you expand on what those are? The ecosystem was the worst part of using Go when I had to, specifically because every tool I vsme across I had to double check if it was compatible with Go modules (and half the time they weren't).

Spot on!

> HN upvote system has nothing to do with right or wrong or constructing a good discussion

Spot on again. You are probably getting a hundred downvotes :(

> I would say that if the compiler got 100% slower, that would be a failure.

Still a weird thing to care about. If the compiler were 100x slower, that would still be faster than continually handling boilerplate with human brains.

Well for that purpose we already have Rust/ Swift and so many other languages. What is Go's USP apart from fast compiler.

Swift is nowhere outside of mobile space, on server side it's pretty much dead.

Native CSP? That's what got my initial interest. The tooling really helped it stick though.

Swift and Rust both are known for long build times.

Mostly caused by LLVM.

Other equally expressive languages have multiple implementations, including REPLs, with Go like compilation speeds.

So Swift and Rust just need equally LLVM fat free implementations for the developer loop, leaving the LLVM backend for the blazing execution release builds.

LLVM isn't fast, but there are other slow parts of the process unique to Rust [1].

[1]: http://gistpreview.github.io/?74d799739504232991c49607d5ce74...

Readability. I never ran into go code I couldn't read.

Is it though? I work on a system that builds in singe to double digit minutes. The productivity loss from x2 slowing down of the compiler is going to be so painful compared to the gains I'll get by not copying some sorting functions etc.

Have you ever worked at a place where builds can take multiple days?

If not, don't speak so hastily about compiler speed not being important.

Slow compilations means work arounds like batching changes together for tests on commits, which means doing binary searches across commits to find out who broke the tests, or just reverting everyone's commit and telling people to resubmit and hope it goes through next time.

Ever waited a week trying to get a single change committed because you kept getting batched with other people's broken code?

Slow compilation times matter, a lot. C++ is the nightmare case scenario for this, I've seen templates that take 10 minutes to compile, and once a team gets sizable, build queues became a thing, and they slow development to a crawl.

A 100x slower compiler has a huge impact on productivity. Copy and pasting code for those instances where you need to use template/generic programming is a huge win over waiting days for any and all changes to compile.

FWIW typically in these situations local rebuilds are faster, from 10 - 30 minutes in my experience, but I've only had the pleasure of working on code bases of such size twice, so who knows how bad it can get.

The way I like to describe it from a systems or operations perspective is this:

Slow (or flaky) build processes create gamblers. You don't want people gambling with prod.

When it's expensive (time or attention) to verify your work, people start lying to themselves. They tell themselves they've done enough, or that red test or wrong data on the screen is "just a glitch", and you know they do this because when they get caught breaking stuff they will tell you the same thing they told themselves.

The progression that works is this:

You make the tool.

You make the tool reliable.

You make the tool straightforward.

You make the tool cheap.

You make the tool mandatory.

People keep skipping step 4 and thinking rhetoric (even verbal abuse) will save them. If you meet idiots all day...

What types of software and systems have you worked with that take on the order of days to compile?

1. Windows Mobile.

2. Microsoft Office.

Windows Mobile was pre-ssd, but we were surprised when we did get the first generation of SSDs and they weren't an order of magnitude reduction in compile times. (They helped but not as much as we hoped for!)

Day long compiles used to be routine. Any c++ project in 2001 had a good chance of taking that long.

I worked on a c# project in 2011 that had 18 hour builds.

I worked for telecom company and our codebase for embedded board with maaaybe 500k LOC of c/c++ (no tmp or any of that crap) took like 4+ hours. That was 2007ish. Didn’t help that our build machines were Sun garbage boxes

You also end up writing patterns like Pimpl[1] into your codebase, which are really just contortions to get faster compilation speed at the expense of readability, maintainability, and (some) runtime speed.

[1] https://herbsutter.com/gotw/_100/

> A 100x slower compiler has a huge impact on productivity

Very true, but the article didn't mention a 100x slower compiler, they mentioned 100% (2x) slower one.

Technically that's a quantitative difference of degree, but it's so big as to be talking about qualitatively different developer experiences.

True, although the poster I was replying to mentioned 100x.

Even at 2x, you can only 2x compile times so many times until things get unmanageable!

Other than C++, and perhaps Rust, all major industry languages compile fairly quickly. As far as I'm aware, this discussion is moot.

All the way back to 16 bit home micros days.

Go's compile speed is its raison d'être. It straddles the line between convenient scripting languages and heavy compiled languages.

You deal with boilerplate using either interfaces or code generation. It's not as pleasant as first-class generics, but it's not completely awful either.

Reply on news.ycombinator.com