Hacking Go's runtime with generics
dolthub.com
dolthub.com
Go GC creates problems and they are non-trivial (as all memory management) but it's not so twisted as in JVM.
Examples:
- DGraph: migrated from RocksDB (C++) to Badger (pure go, developed themselves, based on jemalloc)
- CocroachDB - pure go for storage engine
- go 1.20 proposed feature: package `arena` for region-based memory management
Seems like this: https://www.cockroachlabs.com/blog/why-go-was-the-right-choi... and more specifically this: https://www.cockroachlabs.com/community/tech-talks/challenge... would be good to explore for how cockroach has made it work w/ Go+GC.
Sure, but premature optimization is the root of all evil right? Get something functional, then make it performant by handling the problematic cases. GC doesn't prevent this progression, it's just a significant help to getting something functional.
GC can sometimes lead you into designs that can't be optimized without a change in abstraction, but it's not at all obvious that you would have landed on the right abstraction if you didn't have the GC to begin with anyway. I think getting something working as fast as possible lets you gather the data you need to make a performant design, and having to handle all of that complexity up front without a GC just delays the data gathering stage.
It is not premature optimization but an intentional design choice, and a valid one to make.
This is mostly a myth.
Making sure they're aren't a bunch of alloc's or GC spent in the hot path of a high performance library is hardly a premature optimization.
If you're tackling problems in DB land changes are you already have a fairly clear picture of what needs to happen and have a list of shortcomings you're trying to avoid. Memory layout, buffers, wals, etc are all things that should be accounted for upfront.
You often don't know the what the hot paths are.
I am surely missing something. I thought the builtin map used hashing too?
The maphash package in the standard library (hash/maphash) provides routines that hash string and []byte.
The builtin map type is a hash table.
I should get some sleep, thanks
The main difference is, it is an hardware accelerated & optimized compiler function, now exposed via the new maphash package in stdlib.
And it is fast. Faster than the former performance leader xxh3(avx2/avx512).
Try maphash yourself (example data dedupliction) https://news.ycombinator.com/item?id=34091206
I'm not sure making all comparable objects in Go usable as map keys is a good idea either.
Was a good idea since, it's part of the spec-definition of Go:
> A map is an unordered group of elements of one type, called the element type, indexed by a set of unique keys of another type, called the key type. [...] The comparison operators == and != must be fully defined for operands of the key type
TFA is merely reusing that.
> The proposal argues that Golang developers are forced to work around language when writing hash-based data structures like Tries or concurrent hash maps. Developers who want an efficient hash-based data structure have only the builtin map to choose from. When this issue was originally created in 2017, Golang didn't have generics
And then the author goes to tell us about how this is not a stable solution.
I get wanting to solve a problem, but at what point into fighting the language, to accomplish something simple, do you take a step back and just pick a better one?
- A pretty simple, if not very expressive, language (it's anti-Perl, and seems simpler than JS); you can learn it down to a reasonably productive level in a weekend. (I did.)
- A memory-safe language (like Java) with good performance (like Java) without the need to install any runtime, and the ability to ship as a single binary (unlike Java).
- A reasonably good and ergonomic concurrency approach (like JS, unlike Java), based around channels, not futures, and able to embrace multicore parallelism (unlike JS, like Java, and somehow like Erlang).
- Highly opinionated, well-crafted built-in data structures (like Python), and a general approach discouraging you from rolling your own.
- Good cross-platform support; fast compilation.
That is, it's a language that works well in a case where most of your team is junior devs, and you write some kind of a network server. The devs are forced to follow simple, if repetitive and wordy, patterns, and are limited in their ability to raise creative mayhem in the code base, all while writing reasonably performant, concurrency-friendly code that's a piece of cake to deploy. Combined with some overview from more senior folks, this should work reasonably well (and apparently that's what how it works at Google).
No. it's not a joy to use, it has warts (or, rather, deliberate compromises) in many areas, it requires boilerplate which pushes the user towards copy-paste programming, etc. If you can afford to not use it, don't, but sometimes it's the easiest tool for a job, much like, well, PHP.
Possible with GraalVM. Has been possible if you were willing to pay long before that. GraalVM produces statically linked binaries with only one external dependency — zlib, which is preinstalled pretty much everywhere. Startup time is measured in milliseconds.
I recently built a mid-size REST backend with quarkus — it ships as a single binary with size comparable to what I'd expect to see if it was written in Go, and starts in about 200 ms (from executing the command to when it's ready to serve its first request).
Having written code in both, I hope GraalVM buries Go in the long term, but in our hype-driven culture it's unlikely.
https://quarkus.io/guides/building-native-image#configuring-...
https://github.com/oracle/graal
> A reasonably good and ergonomic concurrency approach (like JS, unlike Java)
This is also very close to being solved.
I like statically typed languages, and go makes defining and using types so simple and straightforward that it feels like you get a lot of the benefits of dynamic typing without the headaches.
What bothers me in Go is mostly the control flow around `result, err := foo(); if err != nil { return err };`, and still non-genericized standard library. But after 1.18 I started to look at Go as a potentially good language, in areas where using Rust is too cumbersome, and using Typescript or Java or Kotlin is also inappropriate.
One can go job hoping as alternative, but then good luck jumping jobs everytime one has to deal with stuff we dislike.
Thankfully I seldom have to deal with Go anyway.
As post 2000 language, its design isn't that appealing.
The language would have been as mainstream as Oberon and Limbo were, if the authors weren't working at Google.
Yes. I have been working with Go for close to a year now, and I still don't get why we use it, if not for the inertia of times past – and I still don't get why it was chosen in the first place.
If we wanted a fast, down to the metal language, Rust would IMHO be better.
If we preferred a higher-level language with a Gc, Java/Kotlin/C# would be, IMHO, much better.
And both paths offer the same level of tooling, so it's still a mystery to me.
Writing code in Java\Kotlin requires quite some investment into the language and tooling (at least that's my Java impression).
With Go - you can start getting the job done from day one most of the time. Some parts of Go may be harder than the others, even challenging maybe, but not as hard as Java.
Lack of OOP (at least in Java's traditional scence) also helps quite a lot (but that depends on project of course)
However, I would not weight too much the time-to-first-makefile of a language if a project is supposed to live for years.
Just put a .netrc file in your homefolder with your username + token and use GOPRIVATE env variable. No need to change any git settings.
After go modules became mainsream most go packaging issues where out of question, imo. At least for me the 'private repo packages' was the only issue.
I'm pretty sure that there are millions of js projects out there, many of them been there for years and will be there for many years to come, all despite npm and js packaging issues in general...
At the same time Go provides enough stdlib to forget about dependencies completely if you are afraid that use of some of them (or packaging system in general) may result in some negative scenario.
This is where Go shines: it can offer enough to be powerfull without becoming java. But without a doubt Go has many things to improve too.
Lol yes, let's just put sensitive information in plain text in my home folder and change the machine-wide configuration, because Go guys decided that if it worked in 1970, it is good enough for today.
> the 'private repo packages' was the only issue.
That's quite a big one for a language targeting companies.
> I'm pretty sure that there are millions of js projects out there
Sure, if your gauge is the JS ecosystem...
> At the same time Go provides enough stdlib to forget about dependencies completely
Go stdlib is decent, that's it. It does not even has basic data types such as sets (no, map keys are not a decent replacement for a set), queues or stacks.
That't just your login and a token that gives read-only access to some repo. Hardly an issue imo. At least if this is your work PC. (also - while home folers is the default the file can be places somewhere else)
>Sure, if your gauge is the JS ecosystem...
Why not? Because some guys on HN is salty about it? JS is extremely popular and widespread. Much better gauge than something being used by a few guys.
>It does not even has basic data types such as sets (no, map keys are not a decent replacement for a set), queues or stacks.
I'd argue that channels are your queues. Maybe sets and stacks will be added now that generics are part of the language.
Like, there is plenty of flaws clear to see in Go but it does some jobs well and that's all that's needed to be useful language.
If you don't like Go (or Rust or Haskell or Elixir or Ruby), that's fine. Just don't comment.
Note I say easy and not good, or simple.
Rust, Haskell, JavaScript, all get caught up on some aspect or another for me.
The executables may be a bit larger (I’m guessing).
When I was C# developer I welcome the addition of FP like feature, but now imho the lang have become very cluttered. GO lang is in another level of simplicity and it is not comparable with C#.
C# is really reliable language, great tooling and documentation but it does not change the fact that is cluttered and no longer simple (which was the point of my comment).
Also I can't see golang going to the same direction, by design go is a lot simpler.
Everything you listed help programmer to keep own sanity.
It's all compromises but they're mostly reasonable compromises.
Whenever I see someone say it's a bad language because X, I seriously wonder if the asker just writes terrible code, or if there's whole swaths of domains I've not encountered. Or maybe a little of both.
I've never once been stumped by a lack of generics, for example, despite how many people claim it's unusable without them.
It's the same everyone. I, for example, never felt I was missing something until I learned about type providers. Now that I don't work in a language that supports them, I hate it. If I had never learned about them, I would probably never miss em.
Although I think you are on to something per the original question about the appeal of Go. For all its faults, Go also gets a lot of things very right that aren't obvious until you have gained the necessary level of familiarity.
Once you rise above 'blub', it is indeed hard to go back. While I spend most of my days in other languages full of all kinds of fanciful features, I did a short stint on a Go project and now especially hate not having blocking functions everywhere (async/await is one of those fanciful features I have to deal with) and sensible error handling.
I mean, isn't that the same thing? You don't know you are "stumped" if you never experienced it. That's what I mean by "feeling the pain".
> Generics aren't exactly magic. They can improve code, but you can live without them if you really have to.
Of course - that's true for essentially any language feature. We can all use assembly if we have to, but I think that's missing the point of why OP asked.
> Although I think you are on to something per the original question about the appeal of Go. For all its faults, Go also gets a lot of things very right that aren't obvious until you have gained the necessary level of familiarity.
Sure, I mean: if you always used a language with an unergonomic and non-integreated builtsystem, you probably won't feel the pain or won't feel it as much as when coming from golang. That would be blub the other way around. Worse: the more languages (that are actually different) you have used, the more unhappy you are with every single existing language.
Not at all. The claim about not being stumped was made in response to criticism that Go is unusable without generics. Said criticism was probably made up, but I guess it was still interesting enough to garner the response that ultimately brought us here.
> That would be blub the other way around.
Seems like blub the same way around. Indeed, the blub paradox itself is paradoxical.
> the more languages (that are actually different) you have used, the more unhappy you are with every single existing language.
And the more you really appreciate just what Go has done by taking an engineering approach and not an academic approach.
At least that has been finally fixed.
For example I deal in ops stuff so pretty common task is "here are desired and current set, generate list of add/delete operations to get from one state to another".
I wrote iteration of that function a bunch of times pre-generics, eventually being just copy-paste-change-types but now I can just write [1] one that compares a slice of types and returns the elements present in left or right and [2] another that can do similar difference on disparate types via conversion function.
Then there is a whole slew of map/reduce and math stuff that now can just have `Map` or `Max` function without worrying about types. Hell I can "just" write "parse all the slice elements in parallel, keeping order [3] once and never worry about it. That's also code that I wrote more than once before generics
* [1] https://github.com/XANi/goneric/blob/master/slice.go#L77
* [2] https://github.com/XANi/goneric/blob/master/slice.go#L106
* [3] https://github.com/XANi/goneric/blob/master/parallel.go#L15
But I think the actual hate isn't coming from Go technical shortcomings. It's language authors and community attitude towards people asking for obvious QoL improvements.
Generics are a bad enough example of community for years tactfully flipping off everyone asking for them and then, hey, here they are and the sky doesn't seem to be falling. But it's not the only example. Just a couple days ago I was researching if there's a way to somehow build struct tags from constants or use interpolation and sure enough, there's an old bug tracker suggestion entry that got a two sentence "won't do" reply with zero reasoning and a bunch of following entries just marked as duplicate and closed without reply at all. This happens a lot. And people don't appreciate it.
Heve you never written a stack structure (since Go doesn't include one)? Never used a min/max function between numbers (go doesn't either)?
Generics make it oh so much nicer to do some stuff but lack of them wasn't exactly something that I missed, just that I wished some libraries had for the cases you mentioned (generic map/reduce/math/etc.)
My guess is that they got excited at the chance of doing something mildly interesting, beyond the mind numbing "regular" go.
..coz they already have code in it ? Coz they are happy with remaining 99% of the language ? It's not hard to figure out cmon.
If people dropped language at slight inconvenience we'd be out of languages
I am an amateur dev developer and used to use untyped languages (Python and JS, mostly).
At some point Python was annoying in docker environments (especially for APIs) and I gave a try to Go. Initially I hated it, then learned to like it more and now I kinda like it.
- typed (this is a game changer)
- compiles to a single self-contained binary (easy with Docker)
- great for APIs without the worries of gunicorn and whatnot
I also discovered TypeScript to help with JS.
So the forst one would be "typed language". I have Python for the rest ("the 2nd best language for any task") and since I am an amateur dev, I do not have the time to dig into many languages to find the pearl.
I had done that myself as an exercise a few years back and had to extract the hash function from the runtime that is used for maps.
In any case, the performance was terrible and code brittle. (lack of sumtypes).
Since then, I've learnt to not bother too much and just use the builtin maps. Never had an issue for my use-cases.
The reference to Tries came from the original issue in the Golang repo about possible uses for a standard library hash function: https://github.com/golang/go/issues/21195
And you're right, the Trie implementation linked there was indeed a Hash-Array Mapped Tried: https://pkg.go.dev/github.com/lleo/go-hamt-key#Key
One doesn't need Go to not overcomplicate things either. I wanted to advocate in favor of Go for a long time, and still do recommend it, but it's the mindset that is more important.
What makes you think they haven't tried what you think are "better ones" and moved away because they had more problems to fight the language/ecosystem about with those?
Such as?
Go's niche includes being native. If you discard that, then virtually anything that is "easy" could fit it.
While one can excuse themselves that they were commercial, GraalVM native image, and OpenJ9 offer free beer alternatives.
However, PTC and Aicas are still in business,
https://www.ptc.com/en/products/developer-tools/perc
https://www.aicas.com/wp/products-services/jamaicavm/
IBM's commercial one is now freely available on OpenJ9,
https://www.eclipse.org/openj9/docs/xaot/
Finally, if you are using a modern Android phone, an AOT compiler is in the box since Android 5, and it was modified into a mixed JIT with AOT compilation on rest since Android 7.
GraalVM happens to be the evolution of MaximeVM, and certainly not the only game in town.
Or is the niche not being able to understand JVM languages, that expect developers to have PhD skill levels, as per Rob Pike's own words?
They would at least know how to make a language that's not a pain in the ass to analyse, which is probably in its favour.
I see you've yet to meet Scala.
> they're not tied to one particular IDE since LSP took off.
There's at least one language server for Kotlin.
I'm not even a interested in (let alone a user of) Kotlin, but you seem to have a very not-objective view of it for some reason.
> There's at least one language server for Kotlin.
I didn't say there is none, I said that any modern language has one. An IDE is not a selling point anymore, in my humble opinion.
> you seem to have a very not-objective view of it for some reason
I just think that shoehorning Kotlin into "the go niche" is absurd , and that the "good code analysis" argument is moot.
That's what you say, but you don't really tell us why.
You're the one asserting that it's unsuitable. So it very much is.
> If you think something like compiling kotlin instead of running a JVM is akin to compiling with go tooling, then at this point anything that somehow compiles to native (and beyond) is in the "go niche".
I don't know about the other commenters, but that's literally the only criteria you've deigned offer so far, aside from some sort of conspiratorial implications.
Look, this is getting ridiculous. Go offers easy+fast tooling out of the box. Any JVM language, compiled or not, will never be anywhere NEAR go's tooling in those terms. The extra compilation layer just makes it actually much worse.
And I don't even find go particularly appealing.
- Elixir
- Rust
- Clojure
- Crystal
In that order.
EDIT: I'd also like to add that Elixir and Rust aren't "exotic." Crystal may be, but it's basically just compiled/typed Ruby.
I've been using Elixir professionally for seven years across multiple companies:
1. Divvy.
2. GoSkip.
3. Podium.
4. Actiphy.
* Clojure is a really nice and simple language, but the tooling and documentation comes nowhere near go, also it uses JVM, which would already disquilify as go replacement.
* Rust is nice (or even better in a lot points) but a lot harder. Concurrency in go is just lot easier.
* Elixir, I agree, although I don't have much experience of the language.
* Crystal, not clue about this lang xD.
The biggest benefits of Elixir over Go for me are:
1. Pattern matching.
2. Easier concurrency (with less resource usage).
3. Piping.
4. Easy to hire and train Ruby or Crystal developers.
5. Phoenix is an outstanding web framework for Elixir.
6. Excellent tooling.
7. Supervision trees.
8. Actor model.
9. Elixir's concurrency is done in private memory, whereas Go uses public memory.
EDIT: Fixed formatting.