Go Type Parameters – Draft Design
go.googlesource.com
go.googlesource.com
I worked on the compiler in 2011 as an open source contributor and firmly got this impression back then which seriously put me off and the absolute farce of how modules were handled (community heavily involved put a ton of work in then out of nowhere Russ Cox just did his own thing anyway) suggests to me nothing has changed.
Generics have been heavily discussed and desired from the beginning of the language and many have contributed thoughts and concepts. At first the designers said more or less 'nope not needed' which changed to 'if somebody suggests a good implementation we will do it'. But would they really? If this design document had come from outside the go team would they even give it a glance?
I like the language and the team are clearly talented but it is a pity that the project is quite so proprietary.
The rest of the community can help with some (not basic) tooling and libraries, at best.
While it's might be true that most big changes come from the core team, open source doesn't mean that they have the obligation to merge (and support) extra features they don't think is necessary.
Simplicity doesn't just happen, it happens because they reject code that increases complexity and would need to supported and maintained.
Projects like the Linux kernel are equally highly selective and reject code that they don't like big or small, but they do at least sometimes accept significant changes.
As I responded in another comment, I accept that might be necessary to maintain the approach of the language but the go team seriously misrepresent this. That is my main issue with them.
How many people have put a lot of work in only to find out that actually the go team aren't accepting significant patches? Look to the go modules fiasco for a good example of this.
Honesty matters.
Only if nothing in userspace (the equivalent of "[Go]'s users projects") has started to depend on it. A OS kernel is perhaps slower than a programming language to see uptake of new features, but the dependencies are still there.
Either they should say 'nope the BDFL accepts nothing significant except from the go core team' or they should change their ways. You can't have both.
Obviously other languages do manage the balance - eg. python, rust, so it's possible but I totally accept that given go's minimalism and opinionated approach they might not want that. But be honest about it.
And good luck reverting a released kernel buried on a server :) especially once users start relying on e.g. a syscall. You might find that it's not quite as different as you want it to be...
Even after migrating to a new executable format, Linux still has optional support to load the old format some 25 years later.
In comparison, Go does breaking changes from time to time, but it is also a much younger project. In general, comparing a language syntax with an api is hard to draw meaningful conclusions from.
Could you give some examples? I would say go has a very high standard of backwards compatibility (to a fault, even. Many good changes are not merged because they could break existing go programs.)
It is absolutely an open source project. However, it is closer to Cathedral style development rather than Bazaar [1].
[1] https://en.wikipedia.org/wiki/The_Cathedral_and_the_Bazaar
Most well-known projects operate in the bazaar model and the Go team claim to also but in fact are operating a cathedral.
> I like the language and the team are clearly talented but it is a pity that the project is quite so proprietary.
To me it seems like nature of things. People who are talented and in control are just not going to take any contribution come their way. For people outside if their thinking aligned with core team they will feel welcome and others will feel unwelcome due to high handedness of core team.
Go in the meantime is chooching along gradually, trying hard not to do anything too big in one go. Modules was the big thing this year, and people are still adjusting and transitioning. Generics MIGHT be the next big thing (idk if the draft design is finished, approved, and work on implementing it is underway). But they spent years considering, humming and hawing, objecting, polling and reconsidering this particular one.
They could have pushed for it much harder and much earlier on, but they wanted for the language to settle first, for use cases to be discovered, etc. They learned from the past, e.g. Java's implementation, and how it added so much complexity to the language spec, compiler, and all the build tooling - which they really wanted to avoid, because of the use cases of Go as a language and the problems it wanted to avoid (including complicated and slow compilation).
How is that true about Rust? Mozilla fired people working on Rust. Now there's a foundation and from the looks of it there are lots of contributions from individual contributors coming through GitHub.
More specifically, which of the various systems are you referring to? (listed here in chronological order)
* Cargo
* rustpkg
* cargo-lite
* Cargo
(yes, there have been two different Cargos....)
>For people outside if their thinking aligned with core team they will feel welcome and others will feel unwelcome due to high handedness of core team.
No, whether you align or not major contributions are not accepted. See the godep vs. go modules debacle for example.
And again, that's all fine with me, and I can see why they'd operate that way, but they claim not to which is my major beef (and allow people to work very hard at contributing without realising it's not going to get them anywhere).
For language changes, don’t they have a formal proposal process where someone submits a spec and the the team (and community) provides feedback before changes are accepted? If you accept that’s reasonable it explains some of how things have happened with dep and generics.
If I remember right dep was becoming a de-facto standard without ever submitting to the proposal process where the Go team could address their technical differences. Obviously there was a messaging failure here and they could have emphasized that it was not formally accepted, but it’s tough to tell people not to use a major improvement to the status quo when there is no alternative to point to and hasn’t been for years.
For language changes so far there’s been almost zero changes approved, by anyone, including the core team, so it’s hard to make a comparison. But if you look at the current generics proposal it’s an enormous amount of work to completely specify. As far as I know, there have been zero actual proposals by the community and several by the core team (most of which were rejected). I think the community sentiment has been more that we should have generics, or “wouldn’t this feature be useful” on the issue tracker without fully fleshing it out, not actually making formal proposals and getting shot down.
I'm pretty sure there's been several different proposals for generics. And I seriously doubt anybody external to the go team submitting even an identical draft would be listened to. Lots of discussion documented at https://github.com/golang/go/wiki/ExperienceReports#generics e.g. https://docs.google.com/document/d/1vrAy9gMpMoS3uaVphB32uVXX...
Even the tracker issue at https://github.com/golang/go/issues/15292 states 'he intent is not to add generics to Go at this time, but rather to show people what a complete proposal would look like' - does that sound like they're open to an external person proposing it?
I saw it even on a small scale back in 2011, back then I knew the generics thing would be solely implemented by the google guys.
There is clearly a 'not invented here syndrome' going on which is entirely understandable actually for a programming language, I just wish there was more transparency on it.
The second a core language change actually involves a person external to google or the team drop the concept that that could happen I'll withdraw my criticism.
It looks like from that thread that the Go team was never on board with certain technical aspects of how dep was implemented. Obviously there are trade offs so it's hard to determine what answer is "right", but opposing a design for technical reasons is still pretty different than pure "not invented here syndrome".
The dep author in that thread says part of the reason he did not make a proposal was he was worried "what negative community consequences (wasted work/goodwill) a misfire proposal might have." But since dep was becoming a de-facto standard, skipping the proposal process was basically doing an end-run against the technical decisions of the Go team (and others in the community not already aboard the dep train).
I have to admit I'm a bit biased here, I use Go and Go modules, followed closely the discussions at the time, and strongly prefer the trade offs made by the (now official) Go package manager over those made by dep. I can understand how people with different priorities (or who dislike Go in general) might feel differently, but I think it's a bit unfair to call rejecting something over real technical trade offs "not invented here" thinking.
> I'm pretty sure there's been several different proposals for generics
I don't know of any? By formal proposal I think they mean complete specification of how it would work with the existing features, not just the start of an idea, because of course you need that to actually add it to the language.
I think few people expected someone external to the Go team would do it, but I think that's more because it's an enormous amount of work that an expert would have to put in for free. In the end I believe even the Go team ended up asking for help from academic specialists to work out the details. [1]
> The second a core language change actually involves a person external to google or the team drop the concept that that could happen I'll withdraw my criticism
They've made almost no changes to the language so far so obviously the evidence is still out on the former case, but for the latter I believe Russ Cox's check/handle proposal [2] and Robert Griesemer's try proposal [3] (both core team members) were rejected after community feedback.
[1] https://arxiv.org/pdf/2005.11710.pdf
[2] https://go.googlesource.com/proposal/+/master/design/go2draf...
[3] https://go.googlesource.com/proposal/+/master/design/32437-t...
I think it's first important to note that Russ Cox himself admitted that the whole go dep vs modules thing was poorly handled which really does back up my point of view - https://twitter.com/_rsc/status/1022588240501661696
> looks like from that thread that the Go team was never on board with certain technical aspects of how dep was implemented.
Note that the package management committee were _regularly_ meeting with Russ about this, e.g. https://twitter.com/_rsc/status/1022588240501661696
They were well apprised of what was going on, I am not sure there was much of an end-run at all - https://twitter.com/sdboyer/status/1034893126320771072 - note that Russ was just ignoring the literal package management committee set up to look at this - https://twitter.com/sdboyer/status/1034893509394882562
Look at https://twitter.com/sdboyer/status/1034894397261979649:
'To the extent i was defensive, it was primarily from having to field yet another greenfield idea, after months of silence.'
If the technical lead literally ignores the package management committee set up to look at package management and goes off and does his own thing - what does that say about community engagement?
This is about the largest [officially sanctioned] community engagement on something semi-core to the language and look at what happened.
>I have to admit I'm a bit biased here, I use Go and Go modules, followed closely the discussions at the time, and strongly prefer the trade offs made by the (now official) Go package manager over those made by dep.
I don't even think this is relevant - it's perfectly fine for people to disagree and for Russ to say no, that's his role as BDFL, but to ignore and not engage at all? The committee could have been pushed in the direction Russ preferred rather than going off and doing his own thing. Or perhaps they could have been told 'sorry we have decided to go in another direction so we don't need you any more'.
>They've made almost no changes to the language so far
I was involved in contributing to the compiler in 2011 back when many changes were made both in terms of language, compiler and standard library. When I worked on it the compiler was written in C (!)
So even when that was not true, the attitude was the same (which is what put me off continuing to be involved in the project).
>I believe Russ Cox's check/handle proposal [2] and Robert Griesemer's try proposal [3] (both core team members) were rejected after community feedback.
It's good they listen to community feedback but that's not the same as accepting fundamental or even not that fundamental language or standard library changes.
Other language projects like python for example, have retained a strict attitude to development but had actual community support through small and big changes to both language and library. It is clearly possible to both maintain a strict philosophy AND accept serious patches from the community.
Again I really have no problem with how they approach things but I do have a problem with the dishonesty about that. They should not set up a package management committee then ignore them, or say you'll accept proposals but then only accept core google employee ones.
I think sometimes people will argue back and forth on details but miss the bigger picture - perhaps that's what we're doing here. The culture of a project (not relevant here but also applicable to companies) always percolates down from the top - ruby, python, haskell, for a non language example of course the linux kernel - each have their own culture, attitude and approach that is clearly informed by the founders/core maintainers. The same is clearly true of go.
I look forward to being proven wrong and when a significant standard library or language change occurs (and v2 is where that would have happened clearly) from the community but I don't think it's going to happen.
And again maybe it shouldn't, maybe go is better off run the way it is, but they should be honest - you can't have it both ways, it's one or the other.
>an enormous amount of work that an expert would have to put in for free
Finally - I dislike the gatekeeping idea of an 'expert', especially for a language like go which is so explicitly anti-academic. Surely what should matter is how well argued a proposal is? Unless you mean a go expert but there are plenty in the community who are.
As for enormous amount of work being put in for free, I think many would have loved to have the kudos of being the person who invented generics for go and would have happily done so. The history of open source is replete with people who have done enormous amounts of work for free - gcc for example.
But you could also imagine an employee of some other company being paid to do it, as happens with the linux kernel very often. Again, I really doubt that an 'expert' (however we want to define it) hired by anybody other than google or even as a hobby submitting the exact same proposal would have got the acceptance Ian and Robert have received.
The community will be used as a sounding board no doubt to cover any edge cases they've not thought about but again that's not the same as somebody being able to _submit_ changes like this.
In any case I don't mean to be hostile here - I wish golang every success and it's well deserved, the core team are extremely talented and I hope they go on to greater and greater success.
I feel the dep committee deciding to skip the proposal process was also a mistake though. They might have felt they represented the community, but as a member of the community who didn't support dep I didn't feel represented. People who don't support the general approach are probably not going file github issues in an evolving third party project, but they would have had a chance to discuss it in an actual proposal process where that is expected.
> I dislike the gatekeeping idea of an 'expert'
> I think many would have loved to have the kudos of being the person who invented generics for go and would have happily done so
After the Go team (themselves pretty smart) struggled for years to find a design that was acceptable to them, they roped in the Haskell people in that paper who I expect are some of the top type theorists in the world... I don't think it is any insult to the Go community to say that it's a bit big for a volunteer project (and those Haskell people are technically outside Google). It seems like designing a generics system carefully so that it plays well with the rest of the language is probably more work than the entire v1 of the language to begin with.
> I think sometimes people will argue back and forth on details but miss the bigger picture - perhaps that's what we're doing here.
I guess the big difference in perception is I've viewed the Go team as making decisions mostly based on their technical priorities (minimalism, backwards compatibility, quick compile times, performance, etc) rather than who the ideas came from. I guess this is because whenever I read the technical arguments around one of these issues, the arguments from the Go team most of the time make a lot of sense to me, and when there is a difference of opinion it often comes down to the Go team putting more weight on their already well-known priorities for the project.
I mean, Russ Cox must have wrote 50 pages on the nuances of package management when he came out with his dep alternative. And the end result put a lot more emphasis on minimalism and backwards compatibility than dep did. So I guess that made it seem less like merely not-invented-here syndrome. Although if the end result is the same, which is big contributions being mostly done by team members, maybe there's not a lot of practical difference.
Do you have a second example?
> That may in turn lead to pressure to modify the language to add some mechanism for using an Iterator with the range clause.
Yes please, there’s so much grammar that just refuses to support any user-defined type.
Nothing embodies "the tale of two cities" esque paradox like Go does. If you're a programmer, Go is so simple that it isn't easy to do anything but the simplest thing.
Agree. I dabble in Kotlin and Golang at work. Those languages can't be further apart.
I occasionally need to read Rust code, but it has been a struggle compared to how easy it is to read Go.
That's pretty limiting when writing optimized code, i.e. when (much) faster versions of generic functions are available for specific types.
> No currying
Not sure why they decided not to support this, seems like a fairly logical feature to allow. Perhaps they couldn't figure out a good syntax to express this?
> Don't confuse the term generic as used in this design with the same term in other languages like C++, C#, Java, or Rust; they have similarities but are not the same.
Many critics complained in the past that Go ignores language resaerch results of decades. Thats true for a certain extend but in the same time they are exploring new ways to approach language design problems.
The go language spec is one of my favorite things about go. Is a learnxiny on steroids. In less than an hour, you fully understand the language.
I am curious to see how this functionality will change the spec. Will it double it in size?
To me it always struck me as overly casual and loose compared to other languages, and reads more like a guide.
For example, what does the following C# code print?
UInt32 x = 1;
UInt32 s = x >> 32;
Console.WriteLine("" + s);
var x uint32 = 1
var s uint32 = x >> 32
fmt.Println(s)
“For the predefined operators, the number of bits to shift is computed as follows:
- When the type of x is int or uint, the shift count is given by the low-order five bits of count. In other words, the shift count is computed from count & 0x1F.
- When the type of x is long or ulong, the shift count is given by the low-order six bits of count. In other words, the shift count is computed from count & 0x3F.”
> Interface types with type lists may only be used as constraints on type parameters. They may not be used as ordinary interface types. ...This restriction may be lifted in future language versions. An interface type with a type list may be useful as a form of sum type, albeit one that can have the value nil.
It looks like there is a willingness to do this, but not a clear solution yet.
One can dream...
It is not as stable/mature - but well, its open source.
I don't know how the v repo looked at that time but just in the last 7 days it had 19 Pull Requests authored by 11 different people merged. Along other strong activity indicators.
https://github.com/vlang/v/pulse
That's gotta be the most active vapoware project I've seen to date.
V could maybe use "misleading marketing" at best, but it's in no way vaporware, as if it was, the author would not be able to perform all the things they did in the blogpost.
It's also over a year old, would love to see the author to revisit the claims of V like they did in the post, but with the latest versions.
Yes, minus the pragmatic (mature, ecosystem, libs, communoty, tools, future, adoption, documentation, etc...)
I think that listing the possible types with "type" inside an interface is a mistake. It means that instead of providing builtin interfaces for existing primitive types, Go asks its users to list the primitive types in user defined interfaces. This will not go well if at some point in a future, the Go team want to add a new primitive type because the user codes will need to be upgraded.
And not providing inference when directly referencing a function is a weird choice (see section Type inference for generic function arguments).
> This kind of type switch would only be permitted if the constraint lists explicit types, and only types listed in the constraint would be permitted as cases.
So adding a new primitive type `uint128` to `numeric` would break all that code anyway: they would no longer be exhaustive type switches.
Putting the programmer in charge means they can add support for `uint128` in the library at any time after the language change.
I think type inference for generic function arguments is intentionally postponed to have a more reasonably-sized change going in first. It can be added later, making the explicit type parameter optional.
The solution for supporting < and > — or rather, continuing to support them only for built-in types — feels powerful enough without introducing full-on operator overloading.
If I read correctly, this does mean we’ll still need multiple versions of some common functions like min/max depending on whether the type is a built-in or something else that’s comparable (like bigint.Num or time.Time), but that feels ok to me. I wonder how the standard library will accommodate this — perhaps you just redefine math.Max to be generic, since such a change would be backwards-compatible for the vast majority of uses.
Like other posters, I’d also appreciate support for user-defined types in language constructs like for..range and [ ], but that’s orthogonal to generics and can be added in a later 2.x revision (just need to define a built-in constraint-interface for these).
I've seen some language changes be announced in the past, can't asses on my own how breaking they are, does the book still reflect the language to a reasonable degree ?
They explicitly did NOT ignore the history of programming languages, and resisted the urge of adding generics without fully understanding the problem that it was trying to solve (in the context of Go). See also: https://golang.org/doc/faq#generics
There's a trend going on in the past decade or so where every language goes to adopt features from other languages, adding complexity without actually solving a problem other than "I use this feature in language X, why doesn't YOUR language have it?"
Which lead to pretty shitty decisions, like half implementations of OOP in PHP and Javascript (have some classes, but not access modifiers), functional programming in Java, or every paradigm ever invented all at the same time in Scala.
That's simply not true. Languages like StandardML, Ocaml, or Haskell have already solved the major issues 40 years ago with types vastly superior to what go offers. Since then, nothing has come even close to that level of utility and usability.
Like so many other things in go, it seems a combination of Hubris and NIH syndrome.
So why haven’t aren’t those languages commonly used?
The simple answer is that there’s a lot more to a language than its type system. Go is popular because it’s simple. Throwing generics on the language without a lot of deep thought and time risks adding marginal utility while destroying a large amount of the core simplicity. While I too want generics, I understand that the caution and skepticism of the core team is not “Hubris and NIH”.
I suspect hajile meant that no type system comes close to the utility of that of StandardML, Ocaml and Haskell, not language in general.
Java or C# decide null is bad and try very hard to add Option and non-nullables, but go's creators just decide they must not have a good reason, so nulls everywhere.
The potential problem of things like empty interface escaping the type system are well known, but go's creators decided that everyone else's experience here didn't really matter, so you get not only empty interface, but it becomes the only way to do certain things because of the lack of generics.
Speaking of generics, every popular multi-use language winds up building in generics or tacking them on later. Go's creators denounce all other solutions as lacking.
What makes them think they are so incredibly special that they don't need to learn from the experience of others? It seems to me that the only reasons are basically ignorance or arrogance and I don't think they are ignorant.
> So why haven’t aren’t those languages commonly used?
Languages don't generally survive without a big corporation paying the bills. If Google had chosen to put their hundreds of millions into SML instead of golang, the world would be a very different place. The first company to give such a language a chance was Mozilla with Rust and we can see how popular it has become. You could view Rust as SML with c-like syntax, lifetimes, unsafe options, no GC, extra pointer complexity, macros, etc. A bit more potential power at the cost of orders of magnitude more complexity due to being targeted at a different, lower-level problem set.
> Just to get your argument straight: nothing comes close to the level of utility and usability of StandardML, Ocaml and Haskell?
I would argue that SML is the best of those three languages and the most comparable to go as well. It's designed to be pragmatic, easy for first-year students to learn, and easy for later students to implement in compiler classes.
Go's headline feature (goroutines aka channels) have been implemented in PolyML or CML a decade or two before go existed and they did it in a type safe manner. Go has multi-value returns, but all that means is that it has partial tuple support unlike SML which has both tuples and pattern matching. Despite what you might think you know from Haskell, SML allows mutation complete with safe (and type safe) pointers.
Rather than interfaces, SML has structural typing which allows most of the same features, but with type safety and generics. There's also sum types (basically union types) which are easy to use with pattern matching and guarantee that all union types are handled safely. Unlike go with nulls everywhere that must be checked at your own peril, SML uses option types and nothing is ever null.
All of that said, I don't hate go. I believe the language could keep its existing syntax while adding most if not all of these features and retaining a decent amount of backward compatibility and making code both more terse and more readable. It could solve its generics issue while also solving its other type problems and potentially picking up extra performance on the way.
Sometimes more features can make a thing worse. More advanced type systems may not necessarily make a large group of people more productive in a language, even if they are strict improvements. Go seems to have hit a sweet spot in terms of having just enough features to mostly get out of the way, and be otherwise straight-forward.
You could describe this part of its appeal as "worse is better". If you aren't careful about how you make something like that "better", you'll actually make it worse. I acknowledge this reality, while still personally wanting algebraic types, optional types, generics, etc. like everyone else. This gives rise to an "innovator's dilemma" of sorts for language designers.
(As an aside, if you look at the size of the Go team at Google, you'll probably be able to conclude that it's unlikely Google has put "hundreds of millions" into Go. But this is related to the core point: there isn't a huge list of language features that requires teams of experts to implement.)
As to hundreds of millions, that's easily true. Most of the people on that team make 500k per year at least with some making way more than that (and we aren't including other benefits of costs to the company who employs them). The language is publicly 11 years old and existed before that.
1e8 / 5e5 / 11 gives a mere 18 developers and less than 200 man years. I seriously doubt the compiler, libraries, and extensive tooling took less than 5x that amount at a minimum.
Re: SML types, I think you may be confounding simplicity and terseness. The last time I touched ML was decades ago, so I don’t have a specific opinion here but consider “both more simple and more powerful” could be a contradiction. The lack of power and expressiveness can in fact be part of what makes things simpler, not strictly because of the language but because programmers are forced to do things in a “dumber” way.
And in the actual world of working software, Go seems to be at least as useful as Haskell. Your criticism seems to come from theory, not practice.
I mention these languages primarily because of their use of the Hindley-Milner type system (I could have mentioned Rust, Elm, and quite a few less-known languages too).
I believe, that for new languages there are only two choices:
1. Build a simple language and don't implement generics _forever_. That is fine! Just stick with it.
2. Build generics into the language from the beginning and do it right. If that's too hard for you, then don't create a language in the beginning. Creating programming languages is damn hard. Amateurs will just fail.