How generics are implemented in Go 1.18
github.com
github.com
A few years later we had go generate. Which I argued was a glorified C macro system to get around the lack of generics. Oh and everybody forgot about the single-pass thing, I guess.
Now in this thread there are a bunch of people commenting that they can't wait to combine all their copy pasted code using a generics which are implemented via a tree traversal that actually kind of complicated the idea of what a pass even is (I think memoization could make this linear, but at the cost of memory?).
This is a language that seemingly insists on breaking the wheel. Notice I didn't say reinventing, because in over a decade they haven't actually gotten a working version of the wheel out yet. I mean seriously, Go was released in 2009: if I'm not mistaken, C# had a pretty good generic system already, and there were a bunch of other workable options to copy from less-mainstream languages. Not that copying has helped: they copied coroutines from other languages, but failed to present a coherent memory sharing model (just copy Erlang FFS), which severely limits the usefulness of coroutines.
Every few years someone gets me to try Go out again and I discover, yet again, that it's still struggling with problems that were solved before it existed. It's a shame that this language has stolen so much mind share from more deserving languages.
Go steals the mindshare because it focuses on the important problems that other languages neglect in part or in full: performance, tooling, ecosystem, simplicity, readability, learning curve, etc. Much important software has been built in fully dynamic languages (never mind a 99% type-safe language like Go), but it’s a lot harder to build in a language that lacks important libraries or is prohibitively slow or for whom it is hard to find/train developers, or whose tooling is poor, or which is difficult to read/maintain, or etc.
So yes, Go is now looking at generics because it’s tackled the more important problems. Not because it was the most important problem and the developers were just too stupid to understand.
We are already seeing the transition happening, e.g. Deis Labs:
https://deislabs.io/posts/still-rusting-one-year-later/
> Given the team’s background in many Go projects, we often hear something like this: “Well, what about Go? Do you regret moving to Rust? What do you miss from Go?” Addressing this in the context of our discussion of Rust felt like a smart decision.
Since I feel like educating you,
> Nothing really, Rust's killer application are domains where using a language with automatic memory management and support for value types isn't an option, trying to use it elsewhere, while possible is sacrificing productivity.
https://news.ycombinator.com/item?id=30391431
That doesn't change the fact that many relevant projects in CNCF are now adopting Rust in their reboots.
Deis Labs, which you also don't know who they are.
Linkerd2 proxy was written in Rust, despite Linkerd 2 being written in Go.
The famous rewrite from Go into Rust done by Discord for their microservices.
The Amazon Bottlerocket uses Rust for large part of the user space, as does Fuchsia nowadays.
Any language can be a systems programming language, even Go, as ARM and F-Secure do with their bare metal Go deployments for microcontrollers.
Go's future is assured, no need to panic.
Why would anyone here have any idea who you are? Your username is not informative, and you don't say who you are in your profile. That's perfectly fine, of course, but you can't then expect people to recognize you as being a particular individual.
You either need the low overhead of no-GC or you don't. If you don't, choosing Rust is a mistake.
---
As for your examples, I'm familiar with the Discord rewrite and I looked up the Linkerd2 proxy. Both programs are low-level services that are designed to handle as much load as possible, so yeah they should be written in a non-GC language.
---
> Linkerd2 proxy was written in Rust, despite Linkerd 2 being written in Go.
From [2]:
"The decision to use Rust came down to several factors. First, a service mesh proxy has some pretty stringent requirements: because it’s deployed as a sidecar on a per-pod basis, it has to have as small a memory and CPU footprint as possible. Because most or all of the application’s network traffic flows through the proxy, it needs to have minimal latency overhead, especially worst-case tail latency."
This should obviously be written in a non-GC'd language.
---
> The famous rewrite from Go into Rust done by Discord for their microservices.
> microservice**s**
Nope, just one microservice; at least according to the article ([1]).
From the article:
"In order to get quick atomic counter updates, each Read States server has a Least Recently Used (LRU) cache of Read States. There are millions of Users in each cache. There are tens of millions of Read States in each cache. There are hundreds of thousands of cache updates per second."
So again, the same exact story - stringent latency/speed requirements.
---
[1]: https://blog.discord.com/why-discord-is-switching-from-go-to...
[2]: https://linkerd.io/2020/07/23/under-the-hood-of-linkerds-sta...
People also use JavaScript, Python and Ruby for use cases that ask for AOT compiled languages
Thankfully no one is going to jail for using a language where it doesn't belong.
That said, Kubernetes isn’t a monolith so it doesn’t need to be all-or-nothing, and one component that would likely be well-suited to Rust is etcd.
Would not pick Go to make a website. But it's probably best tool to make a simple HTTP server or command line tool that deals with files or network.
.NET was released with AOT compiler, although it only did dynamic linking (NGEN).
Mono always supported AOT compilation, just like the C# dialects for Singularity, Midori, and .NET Native for Windows 10 UWP.
"just"
Tell me how me, a student in India with a cheap plastic laptop running Linux, could get straightforward access to some AOT compiler produced by some commercial company in north pole or somewhere.
Open source compilers got popular for a reason.
Now we have graalvm, but getting something running with graalvm is not always straightforward. Even with a full framework like quarkus I have had to use escape hatches @RegisterForReflection. Usability matters. And Go was built from day 1 with AOT in mind.
And lastly Go has better built-in / first-party supported APIs for file I/O, networking etc.. Subjective thing, I guess. When I want to make a database backed website, I reach for spring boot or vert.x; But Go wins for CLI stuff.
I don't know much about C# because I primarily work on Linux.
And, to reiterate, usability matters, defaults matter. Cannot compare 'there existed some obscure vendor who sold an obscure feature in an obscure corner of Europe or Canada', with a mainstream programming language sponsored by a big tech having a good-enough default toolchain adopted by many companies and has lot of jobs;
Then again, maybe things changed in 20 years.
The technology for doing JIT caches on OpenJDK comes from J/Rockit, while OpenJ9 comes from IBM Websphere.
From 2000 to 2009, Red-Hat used to have an Eclipse version compiled with gjc.
Hardly obscure vendors.
Commercial offerings are quite obviously not comparable to free ones.
Seems like Java can't really do AOT, even in 2022: https://news.ycombinator.com/item?id=30444454
Now, I say "really", because this is not about whether you can technically compile Java AOT or not. It's about the level of support - in Go AOT compilation is first-class. In Java, you're stuck looking through multiple second party options, some paid, some with caveats, bugs, etc.
First-class vs second-class support makes a massive difference.
As an example, this is like seeing "Go's advantage over java is its autoformatter - gofmt" and responding "Well, Java has autoformatters too!". Technically true, but irrelevant - in the real world, gofmt is used on 90%+ Go projects. The few Java projects that do use autoformatting each use a different formatter with different sets of rules. First-class vs second-class.
Performance just isn't the problem people say it is. I've done a lot of Python development over the last 12 years, and in that time, I've run into lots of performance problems, but none that would have been easier to solve in Go and none that were particularly hard to solve in Python. In most cases you're just offloading onto libraries (i.e. Pandas or MPT). I've dipped into C a handful of times, but that was more of a "could" than a "had to".
I'll give you a point on tooling--Go's tooling is pretty good. There are a bunch of languages with good tooling these days, but compiling easily to a single binary blob is still a pretty winning feature. But usually that sort of stuff is automated out in the first week of development anyway--is that really enough to justify choosing a language?
Ecosystem? I'm not impressed. There are like 10 languages with healthier, more active ecosystems.
"Simplicity" is generally a pretty bad argument, because what's simple is so subjective. My observation is that simple problems are simple in Go, which makes it a great language for demos, but if you actually try to solve complex problems with it, the language completely lacks features with sufficient abstraction to make complex problems simpler. Where other languages you end up spending 20% of time solving 80% of problems, and 80% of time solving 20% of problems, with Go it's more like you spend slightly less, 15% of time solving 80% of problems because of the simplicity, and then you spend 300% of time solving the other 20% because the actual hard stuff is much harder in Go.
Readability: this sort of goes back to the simplicity thing. Sure, go has less boilerplate than some languages, but that only matters for the simple stuff. If you get into, say, a complex networking algorithm, Go code is going to get lost in all the nonsense of making sure threads share memory properly. I won't argue that, for example, Erlang's syntax isn't a bit ugly, but I can explain my networking code in Erlang to someone who doesn't know Erlang fairly easily because of message passing.
Learning curve: beating a dead horse here, but just because it's easier to learn to solve simple problems in go, doesn't mean it's easier to learn how to solve professional-level problems with go, let alone industry-leading problems which actually constitute the competitive advantage of a business.
Go steals the mindshare because people saw Google were using it and mistakenly thought that if Google were using it, it must be good, even though almost nobody has the same problems as Google.
> So yes, Go is now looking at generics because it’s tackled the more important problems. Not because it was the most important problem and the developers were just too stupid to understand.
Just to be clear, I didn't call anyone stupid: on the contrary, I think the developers fell into a trap that is particularly dangerous to smart people: we think that because we're smart, we already know the best way to do things. That's not stupid, it's ignorant. I don't mean that to be insulting although I'm sure you'll take it that way, but I'll say that ignorance is fixable.
It's telling that in your post you just spit out a bunch of buzzwords without any comparison to any other languages, which is pretty much how Go got here: Go developers generally only think Go is great because they've never used other decent languages. If you're coming from C, then you can be forgiven for thinking Go is the bees knees, but that's comparing a 50 year old programming language with a 10 year old one--C has lots of problems, but there are historical reasons for those problems. Go has problems that Gophers don't even know are problems, because they've never spent enough time working in a language that actually solves those problems. And there's no excuse for it, because Go has no historical reason for not solving those problems: a lot of them were solved by other languages before Go existed, and the developers were either ignorant of the existing solutions or thought they knew better (and definitely didn't, in my experience).
I strongly disagree. I've been doing professional Python development for the last ~15 years, and there are some cases where you can call out to C/Pandas or multiprocessing or something, but overwhelmingly you end up spending more time marshaling to C data structures or pickling than you save. Specifically, I've had to port services to use a Spark cluster to offload the processing because even Pandas was inadequate (80th percentile requests would take more than 60s for a CPU intensive task, and they were also blocking the async event loop). This just isn't a problem in Go because (1) single threaded execution is often 100x faster than in Python and (2) parallelization is trivial and (3) you can easily optimize Go by moving allocations out of a tight-loop (no such thing in Python).
> But usually that sort of stuff is automated out in the first week of development anyway--is that really enough to justify choosing a language?
No, I don't think this is a make-it-or-break-it feature, but it's nice. Moreover, there's a lot to good tooling besides static binaries--reproducible dependency management, no need to set up CI to build/publish packages and docs, build systems that don't require scripting or extensive configuration (any Go developer can jump into almost any Go project), dead simple cross-compilation, standard formatters, etc.
> Ecosystem? I'm not impressed. There are like 10 languages with healthier, more active ecosystems.
The way that I think about it is that you're either in the top 5 languages or you're not; it doesn't matter much where you rank in that tier because once you're in that tier you're pretty much guaranteed to get first-class support for APIs. For example, Kubernetes, Docker, AWS, GCP, etc are pretty likely to support API clients in JS, Python, and Go. Java would probably be next and perhaps C# eventually. For the foreseeable future, Rust will probably have to rely on the community to maintain these kinds of packages. The salient point is that Go is in the top tier of languages which get first-class support, not that its ecosystem is head and shoulders above everyone else.
> "Simplicity" is generally a pretty bad argument, because what's simple is so subjective. My observation is that simple problems are simple in Go, which makes it a great language for demos, but if you actually try to solve complex problems with it, the language completely lacks features with sufficient abstraction to make complex problems simpler.
I agree that "simplicity" is subjective, but I disagree that Go is only well-suited for demos. Solving complex problems is pretty simple with Go; however, Go doesn't do well for eliminating boilerplate (local repetition) which is fine because boilerplate isn't where your bugs come from and in most cases trying to DRY up boilerplate leads to code which is much harder to understand (e.g., while a single map()/filter()/fold()/etc is pretty easy to understand, long chains get convoluted fast while a single for loop remains pretty easy to grok (even in Python, we tend to frown on complex list comprehensions in favor of nested for loops). The real value add for Go's kind of simplicity is that anyone can jump into a Go project and write basically the same kind of code that anyone else would write--there isn't much room for personal flourishes, everything is bog standard. Go is bad for "code as art" or flexing your cleverness, but it's great for developing software.
> Readability: this sort of goes back to the simplicity thing. Sure, go has less boilerplate than some languages, but that only matters for the simple stuff.
Per my previous paragraph, Go's readability isn't in reducing boilerplate (Go is famously criticized for failing to reduce boilerplate) but because everyone writes the same code and the code is fairly simple (e.g., no "inheritance vs composition" debates). Further, the ubiquity of gofmt is also pretty amazing. Some languages are catching up with the "standard formatter" idea (Python, Rust, etc), which is great, but nowhere is it as ubiquitous as in Go.
> Learning curve: beating a dead horse here, but just because it's easier to learn to solve simple problems in go, doesn't mean it's easier to learn how to solve professional-level problems with go, let alone industry-leading problems which actually constitute the competitive advantage of a business.
It's much easier to onboard developers to Go than to Java or C++ or even Python. Being able to bring on a new developer without spending days configuring their developer environment, etc is pretty great. Open source projects can attract contributors immediately and businesses don't need to restrict their hiring pool to people with previous Go experience.
> Go steals the mindshare because people saw Google were using it and mistakenly thought that if Google were using it, it must be good, even though almost nobody has the same problems as Google.
This is lazy argumentation, and trivially disproven. If this were the case, people would give up on Go and return to whatever language they came from. Instead, people who use Go for real projects tend to enjoy it or at least have more thoughtful criticisms than "Go is only popular because of Google".
> Just to be clear, I didn't call anyone stupid: on the contrary, I think the developers fell into a trap that is particularly dangerous to smart people: we think that because we're smart, we already know the best way to do things. That's not stupid, it's ignorant. I don't mean that to be insulting although I'm sure you'll take it that way, but I'll say that ignorance is fixable.
I agree, but I think this applies more to Go's critics than to Go proponents. Notably, Go proponents overwhelmingly have experience with other languages, while Go's critics fixate on how Go doesn't let them program in the idioms of their previous language. The people who try Go for more than a few weeks tend to be able to levy more thoughtful criticisms than "Go is stuck in the 1970s"--they can talk about the advantages of standardization even if they still prefer more expressive power, for example.
> It's telling that in your post you just spit out a bunch of buzzwords without any comparison to any other languages
Yes, it tells you that I was commenting from a phone onto an Internet forum and didn't want to make a huge comparison chart in ascii. :) I'm happy to debate the tradeoffs of Go relative to other languages, but it wasn't necessary to make my points.
> which is pretty much how Go got here: Go developers generally only think Go is great because they've never used other decent languages.
This is another lazy criticism. Overwhelmingly Go developers have previous experience in other languages (per the annual Go survey results). In my case in particular, I've used Java, C#, C++, C, and Python professionally and I've dabbled in other languages (e.g., Rust, OCaml, Haskell, etc) on the side. I regularly have thoughtful and articulate debates about the tradeoffs of Go relative to the aforementioned languages, and many of the other Go proponents who post here do as well. Further, I hope you wouldn't criticize Go developers for lacking experience with other languages without yourself having extensive experience with Go.
I believe the reason it has gotten as far as it has without generics is because it did always have them -- but only for the built-in data types (slices, maps, channels). You just couldn't add your own until Go 1.18. I have several medium-sized Go projects that benefit hugely from generic slices and maps, but wouldn't benefit at all from user-defined generics.
Which is something the Go team is behind as well; they added it to the language, but they aren't telling people how to use it, and are letting the community come up with use cases over time before they iterate on it.
There is nothing Go needs to be ashamed on. Its mindshare is a result of a good choice people made at a certain point in time. A better language will win at its own merits.
> Its mindshare is a result of a good choice people made at a certain point in time.
Its mindshare is a result of Google using it, and people mistakenly thinking that if Google uses it, it must be good.
When you have half a dozen generic builtins but reject the idea that builtins are necessary, and can’t even be arsed to provide a working vector type, you’re nowhere near “good”.
Go is a language made for the human factor.
Go generate by the way is mostly NOT used for generics. Check for example Vugu, which compiles UI documents to pure Go code. Go generate is made to incorporate any random external tool without making it complicated for the humans using it. And it makes sense because the goal was not single pass compilation, it was single pass parsing, which is exactly WHY Go generate is useful. It is still true to the exact way this has been defined. Go is simple in ways that matter.
I’m very open to the idea that I’m suffering from a problem I don’t know I have, but right now you’re just asserting that problems exist without actually telling us how to recognize them.
If anything, I think compiler time is worth a lot less than developer time for the vast majority of projects, and if a compiler pass saves developers even a little time, I say add the compiler pass (within reason--don't keep doing this until the compiler grinds to a halt). This is fairly obvious and encoded into the compilers of almost every compiled programming language. But Go is learning this 12 years into its existence when everyone else knew this before Go existed.
As for goroutines being limited: here's a practical example:
1. Write a basic two-person p2p chat in Go and some other appropriate language. At a high level, each user client has a message log, and the other user can write messages to it. Since the arrays are single-reader, single-writer, there's no real problem, just use an array: this is a nice demo of how simple Go is! You don't need all these fancy abstractions!
2. Expand the chat to include a third person. Again, everybody gets a message log, but now more than one person can write to it. Now, your message log is multiple writer, single reader (assuming each user can maintain their own log of messages they've sent so they don't need to read other people's logs to get those--there's an implicit copy here but that's fine). In Go, start over and write a probably-buggy implementation of a SRMW queue because Go is too simple to have that already. In any reasonable language, wrap writes to your message logs in a built-in queue that's already thread-safe. Note here that "any reasonable language" in this case includes Java (LOL!). In Erlang/Elixir/Clojure, you may literally have accidentally supported a third user already and have to make no changes. There are of course other go implementations which are maybe more idiomatic, but none of them approach the simplicity of a language that supports message passing.
About the second part, you seem to mix up goroutines and channels. In a concurrent system, you would need locks for your example, and channels fix this. Goroutines are just the representation of independent processes and can be implemented and ARE implemented differently per compiler, check e.g. how tinygo and gopherjs do this.
Your example point 2 doesn't make any sense, check fan in and fan out patterns with channels. Go supports message passing, that is what channels are. You really didn't try Go and it shows.
My immediate thoughts are:
- This is rather specific. Not all programs should be using complex thread-safe data-structures. Rather, the primitives are in place to build such things, and libraries can be imported etc.
- These more sophisticated data-structures have runtime costs that are a bit less straightforward to understand as compared to arrays. They require either (a) programmers to be aware of the edge cases or (b) a higher-level language where programmers relinquish some control over runtime performance. To be clear, I personally enjoy many such languages.
Overall, your criticism is equivalent to saying "Go is too low-level". That may well be the case for many applications, but that remains to be argued more precisely.
But I can say that it's the same ad with natural languages. No one ever feels that their mother tongue is limiting, until they learn a new and very different language and learn a pattern/style that they cannot apply in the mother tongue.
- Go
- OCaml
- Python
- Lisp
I think I have a reasonably good sense of what each language offers and what it does not. But OP goes beyond saying "Go does not offer some features that other languages offer"; he effectively says that the language causes severe problems. Where are they?
From my perspective, none of the languages you listed allows for good error handling. But that is obviously only my perspective. I would call this a "severe" issue. In fact, all languages that k I know have at least one "severe" issue.
I guess in natural langages this would be the sapir-whorf hypothesis.
Blub seems like a much stronger effect though, at least anecdotally. But that might be because natural languages are rarely compared in terms of features / capabilities, and even more rarely “ranked” along various such axis.
It is not Go that is struggling. It is you. Does literally every language have to turn into Java?
I'm not sure why certain things generate such fanatical community.
Eg. I remember arguing with iOS developer in my first work that smartphones bigger than 4.7" are useful. He claimed they aren't and Apple, as company focused on best experience of user, will never ever produce bigger smartphone, because it's totally useless.
Rather than creating the structure and passing it down the chain to be modified via typecasts as previously.
Pretty much everything else ends up being limited by interface types being unable to reference structure attributes directly, but requiring method signatures. So it becomes more complicated to reference all types that have an 'ID' attribute for example.
With some types I ended up with so many getter and setter functions to enable generics, that it was better to revert back to 'go generate'.
Perhaps it works better with pure container types that can take 'any'.
The first thing to understand is that Go occupies a space lower level than Java but higher level than C. It includes things like explicit pointers and more control of memory layout.
Many of the design decisions make no sense without that context. Go can’t simply copy generics from Java or memory sharing from Erlang, since the design decisions made by those languages would introduce too much overhead in a language intended to be lower level. In a lower level language more aspects of how a program executes are specified explicitly, rather than accepting more overhead or guessing what a complex optimizer will do.
On generics, I think the team’s position has mostly been that it’s a big project and there’s been other priorities like rewriting the compiler in Go, improving the GC, etc. And that in the spectrum of trade-offs for generics systems they didn’t want to go all the way to Java or C++.
I feel like a lot of HN commenters compare Go to higher level languages and find it lacking, which may be a fair assessment for certain problems but isn’t really understanding the niche it tries to occupy.
That was not the initial team position. Their initial position was "why would you even need generics? Please provide us a usecase". They actively denied there was a need for generics.
Do you have a source for this? It doesn't match my recollection. As I recall, the Go team always said that they may add generics in the future, but also that they would also like to see compelling use cases for it. That is not an unreasonable request.
There are also other differences that would prevent using C#’s system as is - Go doesn’t have the reference/value type dichotomy or inheritance. I think they also wanted to be able to abstract across float/doubles etc (unless C# added that recently).
Anyway I like both languages and I’m not trying claim one is better than the other, just trying to explain the design space Go seems to occupy.
There are definitely differences in the languages, but I'm not sure that Go really had anything to figure out or invent. I don't buy that argument.
In C# most objects are 'class' objects, which are implicitly passed by pointer, although some are 'struct' objects passed by value. In Go rather than making the decision once at the type level, the decision to pass by value or pointer is made explicitly every time an object is used.
> I'm not sure that Go really had anything to figure out or invent
If you look at the discussions for Go (which started as early as 2009 [1]) [2] [3] [4] [5] generics seems as big or a bigger project as the other improvements made to the language over the last decade. My impression is there being a broad spectrum of potential trade-offs across compile speed, execution speed, convenience, etc.
The totality of the prior art here is above my pay grade, but I can quote the Haskell people they roped in [4]:
> We believe we are the first to formalise computation of instance sets and determination of whether they are finite. The bookkeeping required to formalise monomorphisation of instances and methods is not trivial ... While the method for monomorphisation described here is specialised to Go, we expect it to be of wider interest, since similar issues arise for other languages and compilers such as C++, .Net, MLton, or Rust
I think C# occupies a fairly nice design spot also (and takes advantage of its class/struct system to get fast compiles but also performance when needed). But it's not like the C# design couldn't be improved on. As far as I know you still can't write math code that works across 32 and 64 bit floats. And array covariance is implemented in a way that isn't type-safe and relies on run-time checks and exceptions. [6]
[1] https://research.swtch.com/generic
[2] https://docs.google.com/document/d/1vrAy9gMpMoS3uaVphB32uVXX...
[3] https://go.googlesource.com/proposal/+/master/design/go2draf...
[4] https://arxiv.org/pdf/2005.11710.pdf
[5] https://go.googlesource.com/proposal/+/refs/heads/master/des...
[6] https://codeblog.jonskeet.uk/2013/06/22/array-covariance-not...
Never used ArraySegments in C#? They exist since version 2.0
Abtracting over float/doubles is called generics.
Java was originally bytecoded, but at some point gained various ahead-of-time native compilers. Did it become lower level when that happened? It did not.
I think when C# and Java people compare with other languages they treat it as a checkbox exercise without caring about how good that feature actually is.
My chief attraction to Go is simplicity. I can be away from Go for years and pick it up again really fast. With simplicity, you naturally need to make some sacrifices, but I think Go team has been diligent about choosing those.
Just so we're clear here, you're using the word "complexity" to describe the situation of not having to copy paste or `go generate` 30 different implementations of the same class for different types because your language lacks generics.
That being said I'm happy to see that Go now attempts to find a tradeoff between generics and compile time, if their approach proves successful hopefully it could lead to improvements for other languages as well.
No programming language is loved only by rhetorical geniuses. Of course you can find some people who've said some silly things about Go or generics. But so what? You can also find people who've said silly things about your favourite programming language.
Sorry, but anyone who's been watching Go's development long enough remembers this: it's not being made up.
Dischord is good if the alternative is consensus around a backwards language that wastes huge amounts of people's time and money.
> Of course you can find some people who've said some silly things about Go or generics. But so what? You can also find people who've said silly things about your favourite programming language.
The problem being that in the case of Go, the people saying silly things are the primary developers of the language.
Making broad brush accusations of bad faith just doesn't help to get to the bottom of any technical issue. It's fine to express your opinion, but if you publicly say bad things about a community, you should be prepared to substantiate those claims when asked.
HN commentary on Go seems to repeatedly return to increasingly exaggerated and vitriolic claims about the supposed idiocy of the Go team or Go community in relation to generics. The factual foundation of these claims is virtually nil. It appears to be some kind of group false memory phenomenon, where vague second hand reports, endlessly repeated, become accepted as fact.
I would therefore invite you to actually try to find some examples of these bad faith arguments, for your own benefit if nothing else.
In .NET on Windows you can sometimes observe this because generic types in your stack traces (in the debugger) will be replaced with 'System.__Canon' [https://twitter.com/matthewwarren/status/1161249300401311745] instead of the actual type - this indicates that the type was completely erased, so the current function could be running for any number of types and the type can't be identified based on the current instruction pointer.
The shared code + blob approach becomes more necessary in an AOT compiled environment like Go (and you'll see it used more in AOT compiled modes for .NET) since you can no longer rely on being able to on-demand JIT some optimized code when a new type shows up.
Sub-dictionaries are described here: https://github.com/golang/proposal/blob/master/design/generi...
It looks like the compiler needs to walk down the entire call tree from the top-level generic and then compute new dictionaries for each called function. Since the compiler may not know the entire call tree, it may have to build nested dictionaries.
Wacky stuff!
There’s a concept called polymorphic recursion, supported by languages like Haskell, which goes beyond that and allows using arbitrary derived types in a function, in particular, in recursive calks.
My interpretation of the section in “non-monomorphisable functions” in the original article is that Go’s compilation strategy doesn’t handle that currently.
GHC allows recursion that is "polymorphic in the types" but "monomorphic in the runtime reps". There is no reason why Go shouldn't allow polymorphic recursion that is "monomorphic in the gc shapes" either, though they might not have bothered to allow it yet.
Go does separate compilation, it's an extremely important property for Go.
Go code can be built as a shared library, loadable by both C or Go code, albeit this is an unusual use case for Go.
However, when multiple Go shared objects are to be linked together, they must share the same Go version, including the toolchain version, so there is no forward compatibility.
So if there is a package in common such that an implementation change could cause something like a cgshape change, well the cgshape change won't matter since any implementation change will change the package's build-hash. And so everything will need recompiled anyway, regardless of whether the cgshape changed or not.
I suppose there's always RPC-like options.
See:
* https://pkg.go.dev/golang.org/x/exp/constraints
I proposed it as a compromise between full monomorphisation and runtime code generation.
[1] http://oneofmanyworlds.blogspot.com/2011/11/draft-2-of-propo...
"We do not propose to change the reflect package in any way. When a type or function is instantiated, all of the type parameters will become ordinary non-generic types. The String method of a reflect.Type value of an instantiated type will return the name with the type arguments in square brackets. For example, List[int].
"It's impossible for non-generic code to refer to generic code without instantiating it, so there is no reflection information for uninstantiated generic types or functions."
In theory, a slice of structs that implement an interface should be able to be used as a slice of that interface. But due to the implementation, that requires a full copy.
You can see it this way: there is implicit syntactic sugar that creates an interface value (pointer + type info) when you assign a value of type T to a variable of type interface I where T implements I. This happens on variable assignment, function parameter bindings and return value bindings.
A slice of Is is just a very different type from a slice of Ts. The assignment magic sugar works, but it works at the level of slice elements, not the whole slice.
That’s a basic fact of type safety that people often get wrong. For example, the JVM famously allows upcasting arrays, with very surprising results.
But even an immutable list cannot be made covariant. Or, say, funcs.
It absolutely should not even if it could (which it can not since the memory layouts don’t match), as that undermines the type system.
Yeah but immutability’s not even remotely a thing in go land.
It looks like there are still a few release blockers [2]. I'd imagine RC is fairly soon though.
EDIT: as mentioned by _fz_ below, RC1 has already released. Seems like the full release will most likely still be in March.
[1]: https://go.dev/blog/go1.18beta2
[2]: https://github.com/golang/go/issues?q=is%3Aopen+label%3Arele...
RC1 was released two weeks ago.
Hasn't this been a very long time coming? Iirc there has been much dispute over this in the community. Can anyone weigh in on what the other options were?
About how well that decision will turn out in practice, we'll start to know soon.
Mind you, C/C++’s ridiculous header system is probably still a much bigger issue. Especially because C++ templates usually need to go in header files.
It's not the monomorphization itself that takes a long time, it's the LLVM codegen due to all the extra code being generated from duplicated function implementations. I've heard macros slow for largely the same reason, it's not the macro execution itself that's slow, it's compiling all the generated code.
If that were true, the same issue would apply to hygienic macros, yet it doesn't. It's distinctly a problem with proc macros.
IDK about that. A project I'm working on is template-heavy in a major way, entirely header-only (~15kloc of this: https://github.com/celtera/avendish/blob/main/include/avnd/c... more or less) but with a barely correctly set-up dev environment with clang and PCH (a couple lines in CMake), ninja and mold, individual rebuilds are pretty much instant ; a complete rebuild which on my machine builds 49 libraries and 38 executables takes a whopping 7 seconds, CMake included.
I read about how .Net does Generics at the VM level and was very impressed, a good compromise between C++ "macro expansion" and Java "type erasure", both of which seem extreme.
At the end of the day, though, when generics are present I stop worrying and just use List<int>, List<float>, List<char>, etc. without a second thought. The compiler, VM or runtime can deal with it although I might get punished in some way.
Modules don't make instantiating templates any faster, it only makes parsing them faster.
As for the comment about MSVC, it's mostly off topic, doesn't relate to the question that was asked (which had nothing to do with which compiler provides the best experience), and even if you accept that MSVC provides the best module experience, it's support for it is still very buggy and not suitable for anything other than experimenting and beta testing.
However, for code generation Go obviously uses registers since way before the 1.0 days.
In general discussion I see around is that all Rust's perf optimizations are to Rust's credit (nothing about LLVM optimization) while all woes around Rust's compile time lay at step of LLVM and Rust is not be blamed.
To pick one of my examples, D templates and compile time metaprogramming are even more powerfull than Rust and C++ ones, yet it compiles just as fast.
So it had them roughly as long as Go existed
> we were biased too much by experience with C++ without concepts and Java generics.
https://go.googlesource.com/proposal/+/master/design/go2draf...
Most people are really bad at estimating the implementation complexity and tradeoffs inherent in various language features. I'd say that C++ serves as a warning to others... think before you add features to your language, or you'll end up a total mess, like C++. Java and C# gave us some interesting and subtle lessons about what does and does not work about language features like generics. C++'s templates are just a total mess, all around. Java's generics are kind of a funny compile-time feature and have a ton of limitations. C# generics have some downright nasty interactions with other parts of the type system, like operator overloading.
IIRC the designers of C# have some regrets about how generics and other features were implemented. This is from an in-person talk, I don't have anything to cite.
You should take almost nobody at face value when they talk about language features like generics, because there is almost nobody with direct experience both designing and implementing those features. You should be pretty skeptical about what I'm saying too, IMO. But do believe that the design tradeoffs for generics are a complicated enough to justify the wait.
The Golang approach seems to strike a balance between extreme approaches. The C++ approach is "pay for what you use" which, in practice, is actually an extremist language design philosophy. In C++ / Rust, you are supposed to pay for templates only in code size and not in runtime. Everything is fully instantiated. The Haskell approach is "one abstraction fits all, everything is a pointer, use an implementation dictionary, monomorphization is an optimization". Again, something of an extremist approach (similar to Java's, but the comparison with Go / Haskell is better because Go / Haskell both use implementation dictionaries).
A middle approach is actually somewhat novel, believe it or not.
I'm sure that's possible in other languages, but the program I was trying to compile wasn't that complicated... It had just been written by someone who believed every concrete class needed an abstract interface it was implementing.
Here for example: https://godbolt.org/z/4nzPY3h57 javac doesn't even bother to resolve that static field at compile time, which wouldn't even have observable side effects. Nope, just blindly emits a static initializer to invoke square(2) at runtime.
Or for a real fun one, enums: https://godbolt.org/z/xjbYvc8dx
Now some of that is necessary to handle all the stuff other classes could do with the enum, but some of it is just unnecessary. Like building the values array, which invokevirtuals all the ordinals of all the enum values. Ordinals the compiler already knew, since it's literally in the same .class file further up (and these are always in the same .class file, they are in lock-step, so there's again no observable difference if the values array was defined with constants instead of invoking ordinals on each field).
And then the trivial switch statement there isn't optimized at all, either, which again wouldn't be observable (compare against what eg. gcc would produce: https://godbolt.org/z/414rGjbE1 )
Further note that these examples are both static initializers, which means the JIT never "fixes" these (and thus instead perpetually contribute to slow startup). So if javac was going to bother with any attempt to optimize at all, you'd think it'd be here.
Bad bytecode is the kludge of byte code you get when you try to write the same thing in kotlin: https://godbolt.org/z/9sGa6csa6
And the bytecode is of comparable quality to javac's.
I remember there was some simple way to crash a Haskell compiler with an out of memory error by defining a series of types, where each successive type required twice as much memory as the previous type to represent.
It's a real problem.
Anecdotally: I maintain a Debian package with a large-ish (but far from huge), template-heavy C++ codebase. I had to give up on compiling on 32 bit machines long ago, simply because not enough addressable memory was available to the compiler. I know other maintainers have to heavily tune the compiler to work around the problem, too.
I was once asked by the Ubuntu folks to disable parallel builds altogether, due to even their 64 bit builders running out of memory altogether.
And this package is just large-ish – it's nowhere near truly large (like a browser or an office suite).
Operators are static methods so why is that surprising or nasty?
Just take a look at the rules for method overloading in C#, or take a look at one of the various C# compilers to see how methods are resolved.
That's a feature not a bug of golang. Major language changes like this by design are supposed to take a lot of time and thought to land.
I'm actually really happy to get generics in golang, and I'm happy with the team giving it as much thought as they need, but we are only gonna get so far within the current paradigm of trying to model the universe from a few text files. Generics are nice, but we shall do better in the future!
But sure, let's not give any ideas or question anything ever again, someone might get offended.
(The answer is very far)