Why Go and Not Rust? (2019)
kristoff.it
kristoff.it
Do people actually find this to be true? Honest question.
In my personal experience, as Go programs grow in size, they become increasingly unmaintainable. Avoiding abstraction and keeping everything explicit sounds great in theory, but what actually happens in a larger project is that they just become a mess. Functions start taking on seemingly-unrelated extra parameters just to shove a tiny bit of extra necessary information four layers down. Details related to one coherent idea are strewn about everywhere because it was easier to add the logic in situ to a dozen places rather than have one part of the code responsible for it. Internal idioms are copy-pasted across dozens of files, many of them missing improvements made to later copies.
I've seen this happen in every large go project I've jumped into, and it (subjectively to me) seems to be significantly worse than in other languages. Maybe it's because—on average—gophers tend to trend more junior in their careers and there haven't been enough senior mentors keeping things well-factored. But at this point it makes me instinctively dread any time I have to switch to a new project that's written in the language.
The whole "complexity is bad, explicit is good" mindset vaguely reminds me in some way of the hype around "schemaless" databases. No database is schemaless. All this means is that you still have a schema, it's just implicit, inconsistent, and now you lack tools to understand and/or manipulate it. The complexity in your go program still exists, it's just scattered absolutely everywhere and you lack good tools to wrangle the worst parts of it.
Of course team quality matters much more than language choice, so my experience will differ from others.
The types of problems I get hired to solve are messy, real world problems. There's not much that you can do with fancy abstractions when everything you deal with is a corner case, so explicitness is extremely appreciated.
I do find Go projects remain understandable even when they get large, and in my field -- distributed systems -- they are often used in tandem with a service oriented architecture that limits the size of any particular component, anyway.
I used to write Python. I don't miss it. Go keeps my coworkers from writing code that's too clever for me to understand, and keeps me from doing the same to my team.. and to myself, in the future.
Until someone whips out `import “reflect”`.
I’ve always found it confusing that people claim Go is simple when it contains runtime type reflection, which is hiding just under the surface of lots of the standard library like `json.Marshall`. Trying to explain to a new Go developer why []int isn’t assignable to []interface{} is always fun.
To take one common use case for reflection, Json marshalling/unmarshalling, if I were writing an application that needed to do this, and for some reason I could not use a library, I absolutely would not use reflection, but would write code that specifically targeted the classes that I was marshalling/unmarshalling. Conversely, if I’m writing a library, I don’t know what those classes are so I must use reflection to manage this. That’s the key difference here.
[0] https://cs.opensource.google/go/go/+/refs/tags/go1.19.5:src/...
In my experience, you can not make complexity disappear: it can be reduced to a point, but there's a domain specific lower bound on any project.
So the question is: where do you move it?
Rust decided to move it into the language, whereas Go prefers to move it into your code. I guess that beauty is in the eye of the beholder, but I stand strongly in the first camp, for the simple reason that I find more complex, but more concise and specific forms, to better carry over the thoughts of the people whose code I'm reading.
On top of that Go is pretty silly on many more minor aspects (unused import & variables are a no-go, but dead functions are A-OK; I have to change my session-wide git settings to access private repos; the formatter is very poor; ...).
Put another way, language complexity will be consistent in every project that uses the language, but complexity in the project code can vary drastically from one project to the next. The tradeoff here is that once something gets put into the language (or the standard library, since effectively they share the same compatibility concerns), you can't easily change it later. People coming to Rust from other languages often are surprised that the standard library seems to be "missing" things like a HTTP client or regexes, and to a certain extent, they're right that it appears to be inconsistent with the philosophy of being willing to allow complexity in the language. I think the misunderstanding stems from the fact that decisions driving the development of the Rust language tend to be a lot more conservative than realize due to the (not entirely undeserved) reputation of the Rust community at large. As a somewhat orthogonal example, there's a common perception that the Rust community wants to rewrite the entire world of C/C++ code in Rust, and while the idea of that certain does excite a lot of Rust programmers, the people who are in charge of Rust have actually put a _lot_ of effort into making interop between Rust and C a viable way of working, and there's a surprising amount of tooling in the ecosystem to help bootstrap wrapping C libraries. On the other had, it's fairly well accepted that Go is not intended to be a replacement for 100% of the use cases of C, and yet almost everyone I've talked to who is passionate about Go has had a fairly negative view of the experience of using cgo. I've really come to appreciate how well-thought language-level decisions are made for Rust, and it's unfortunate that strong emotions in discussions comparing languages (generally on both sides) tend to drown out the higher-level insights that would probably be more useful given the low likelihood of succesfully convincing someone online that your language of choice is in fact superior in some way.
That's another axis: it's how big should be the stdlib on a scale from Python to C.
E.g. Python is much closer to Rust than Go w.r.t. the complexity of the language, and it has a huge standard library like Go.
I think that the simplicity and explicitness actually help to make these kinds of flaws more obvious. This is probably better than hiding poor structure with clever language features. (I’m not saying that those features are always bad — Go is certainly missing some upsides from these — but they can also be used to mask deeper issues.)
In other words, Go’s simple language won’t stop you from seeing bad code, but hopefully means it’s easier to see and understand structure (and improve it?). Maybe the awfulness of a codebase is more a consequence of the team culture and history, but the language can influence how difficult it is to see the problems. I certainly think this would apply to C/C++ (with C being less powerful and harder to hide).
Considering older languages such as C++ and Java, the latter has several frameworks for web apps where a lot of complexity is opaque to the user as long as the user sticks to the defaults. For C++ there are few.
In either case, over time certain patterns of design emerge. Typical concepts such as dependency injection makes sense with some semantics but but less so with others, e.g., with Scala’s implicits, while still being applicable in another style of Scala. With Go, one can alter receivers on a per-package base.
The complexity never goes away, it’s always somewhere and there’s always some cost to managing it. With Spring, you pay with performance as long as you stick to the defaults and they don’t match exactly your model of execution. With Scala and Haskell, it’s on the type level and mostly implicit but the compiler has got more skin in the game and gets to help you a lot—you sort of ask the compiler to assist you more the more explicit you are about your types. With Go, the complexity is scattered and smeared across your code structure. And it’s on you to properly chase it. With generics it’s a bit simpler now. And abstractions are a boon in the hands of an experienced developer. And sometimes they really should be discovered and not imposed—depends on whether the problem is formulated beforehand or in the process of an agile cycle.
There are indeed trade-offs in terms of where complexity lives, but I would not say that with Go it is smeared across the code structure. That depends on design. However, some complexity is more explicit and apparent in the code itself, which I believe the language philosophy holds as a good thing (hidden complexity is worse than complexity). For example, the classic “if err := …; err != nil” is the complexity of error handling in your face. There’s no lovely “!” operator or exceptions to hide that away and produce a beautiful, minimal function. But exceptions are an obvious example of the danger of these kinds of features; they will hide all kinds of unexpected code paths and behaviors. The Go programmer is forced to deal with these things in a “dumb” way, but I think this ends up being beneficial in most cases.
I’ve worked on Nomad and so am far too biased for my opinion to carry much weight but: Nomad and Consul share a lot of code structure so I find both fairly trivial to navigate and find what I’m looking for most of the time.
The complexity of your specific problem distributes itself unevenly across all knobs and bolts of the development process. Some teams tend to manage Python code bases more efficiently, while others might prefer C++ over Rust. Some languages have libraries available that take away a lot of complexity from your immediate control by solving some part of your problem—you delegate it in the hope that it’s managed properly over there.
People who claim that "you can learn Go in a couple of days and just run with it" have no idea what they are talking about. Writing Go is easy. Writing good Go is very hard.
My personal impression is that the Go fans simply assume that Rob Pike and Ken Thompson are language design gods and can do no wrong, but what I saw instead was language decisions and inconsistencies stuck in the decades past, incompatible with the complexities and scale of modern software development.
There are things about Go that are outstanding, like the ease of cross compilation and self contained static binaries making the development of CLI tools trivial.
However the language design itself is too rigid and inconsistent to allow for software engineering paradigms that even the community itself advocates for.
Without highly skilled software engineers at the start, my experience has been that Go projects tend to devolve into spaghetti reasonably quickly.
I feel there are a lot of similarities with the JavaScript ecosystem where we see a lot of bad advice in community posts that people feel passionate about.
I think Go was an innovative language back in the early 2010s when they started thinking of design patterns to improve the ergonomics of concurrency. Since then I have felt continually frustrated by how close it is to my perfect language but how they just keep missing the mark.
I still use it to write CLI tools though. I would use Rust more but it's annoying to cross compile (particularly for MacOS)
Some modern language complexity is abstraction that allow us more complexity while still being manageable.
You are spot on about "schemaless" databases.
I would also add "Patterns".
When the compiler doesn't provide enough features, the programmer has to do some of the work the compiler would do. This extra work are the software patterns.
Functions are a software pattern in assembler. They are a language feature in structured programming languages.
ADT are a software pattern in structured programming languages. ADT are a language feature in object-oriented programming languages.
And so on...
You can certainly have a database of JSON files without imposing any schema on them, apart from perhaps some form of id. Whether it makes sense is a different story.
func (db* Database) GetEmployee(id int) (string, error)
func (db* Database) SetEmployee(id int, value string) error
func (db* Database) ListByDepartment(departmentID int) ([]string, error)
In fact, this is how you have to work when you're dealing with large amounts of unstructured data.
Again, you have a schema. It's just invisible and implicit, and you have no tools to manage, enforce, or deal with it.
When I say no schema, I mean the user has no control over what's in the data. They can query for employee id's, pet names, birthdays, etc., and get back meaningful data or just Nones.
Thinking about a schema this way is useful.
It allows you to e.g. evolve systems in a backwards compatible way without having to update every single component in a pipeline in lockstep.
Most marshalling/unmarshalling systems can't do this.
ie. doesn't matter if Go can unserialize like this if the devs on your team hate the idea
Copy paste? Again all languages.
You are complaining about the authors not the programming language.
Rust tends to solve problems by adding more constraints, which often leads to more complexity. It's great when that complexity is inherent (like with static typing), but quite painful when the complexity is artificial/incidental. In the domains I work in, the healthier approach is usually to decouple more, rather than add more constraints.
For example, in Rust, we often have to use an index or an ID where a regular reference would do in Go. We need to do extra refactoring of function signatures to pass collections down the call stack so we can then use that index/ID... and then find we can't modify a function signature because it's a trait override, and resort to tricky workarounds. In Go, we just use a reference. That choice is decoupled from memory concerns.
Another example is that in Rust, we often need to refactor to add `async` keywords to callers (and their callers) and run into the same problem. Go decouples all that coloring away.
Another example is how Go interfaces are structural, which further decouples a caller from a callee.
Hillel Wayne had a stellar article on how constraints lead to complexity: https://www.hillelwayne.com/post/complexity-constraints/
Go is designed to decouple away details that are unimportant for the vast majority of domains. That means a codebase can change faster, require less refactoring, and generally be healthier in an architectural sense.
That isn't saying that Rust is bad. It's amazing for embedded programming, it's much faster for some domains, and it doesn't have some of Go's other nonsense like nil and how their defer isn't block-scoped. But one can't discount Go's architectural benefits, especially when starting a new project that will be worked on by a large team.
Just my two cents, reasonable opinions may differ =)
This alone is what makes me love go. It makes working with a team on shared code less of a hassle than any other language I've used.
To me, that's worth more than nearly any amount of language features you could name.
Go does have “coloured functions”, it’s called `context.Context`.
Any function in go performing IO usually has to take `context.Context` as its first argument so it’s cancelable. Which then propagates that requirement to the caller just like `async`.
That constraint can be discharged with `context.Background()` just like the async constraint can be discharged by not `await`ing the future in Rust.
Also, changing an immutable reference to a mutable one in can land you in a world of hurt in Rust.
I'm fairly certain that most functions in Go don't take a context. There are tons of helper functions like those in the fmt or strings package that don't, for instance.
I think the level of pain you experience from mutable references in Rust depends on if you’re coming from an OOP or FP background. I have a FP background and so the patterns I use to build code already greatly restrict mutation. You can usually change code that updates data immutably (creating a new copy of it) with mutable code in rust because the control flow of your program already involves passing that new version back to the caller which also satisfies the borrow checker in most situations.
It’s like the ST monad in Haskell; if you modify an immutable value but no one is around to see it, did you really mutate it?
Not true in my experience. You would use context.Background in a test situation. It's also commonly used for short-lived applications like a CLI. You can see kubectl uses context.Background quite a lot: https://github.com/kubernetes/kubectl/search?q=context.backg...
> I think the level of pain you experience from mutable references in Rust depends on if you’re coming from an OOP or FP background. I have a FP background and so the patterns I use to build code already greatly restrict mutation. You can usually change code that updates data immutably (creating a new copy of it) with mutable code in rust because the control flow of your program already involves passing that new version back to the caller which also satisfies the borrow checker in most situations.
There has to be a better solution other than needlessly copying data.
Sorry I don’t think I explained myself very clearly.
What I’m saying is Rust allows you to optimise FP style patterns by replacing copying data with in place mutation because the control flow required for handling the flow of immutable data is also well suited to satisfying the borrow checker i.e. explicitly returning the data back to the caller. Because FP patterns can’t automatically communicate through shared mutable references they also lend themselves well to Rust but without the overhead of copying data because an in place mutation can be used instead.
> Not true in my experience. You would use context.Background in a test situation. It's also commonly used for short-lived applications like a CLI. You can see kubectl uses context.Background quite a lot:
My experience with Go is entirely within the context of API and microservice design where circuit breakers are very important which is why my experience with context propagation may be different than your own.
It’s also not surprising to see context.Background() used within tests because it’s being used precisely as I described above to “discharge the constraint” because you can’t propagate the constraint to the caller because test functions in go can only take one parameter `*testing.T`.
Also I probably see contexts used as much for supervisor/component management as IO. Most of the stdlib IO APIs lack Context support:
- the io and os packages have 0 references to context
- the "net" package has contexts for dns and making connections, but reads and writes don't use contexts
This would do the right thing in the 99% of sequential code, and could be overridable the same way as today for power use cases.
There are TONS of deadlocks floating around 3p library code due to the complexity and confusion around managing deadlines and context-triggered IO interrupts.
Not sure what you mean? The context would be inherited from its parent.
I’m making two points: 1. Context should make its way down to IO, and 2. they should be implicit by default. All for the same reasons Go has automatic memory management - reducing complexity and surface area of bugs for the 99%.
Function coloring may seem innocent in a self-contained example, but become problematic at an ecosystem-level: if one player is not cooperating, the bets are off for everyone else. All dual context impls I’ve ever seen have the context free version simply invoke the other with the background context, indicating a lack of meaningful semantic distinction requiring an API level differentiation. In simpler terms, the act of choosing between either context or no context version is purely a mechanical “do I currently have a context” yes/no determination. This is prime criteria for warranting implicitness, imo.
I hate dynamic scoping for the very same reason I would hate having context be inherited from the parent caller.
Not only would it would result in a function having different behaviours depending on what the call stack looked like at runtime (which is bad enough), but it would be invisible to the reader of the code.
What I like about Go is that it is very easy to visually inspect the code in code reviews.
Imagine looking at a diff in a PR that changes function `foo()` to add a call to function `bar()`, with `bar` using a context and `foo` not using a context.
Do you really want to have to read all possible call-sites to ensure that the context is what is expected when `foo()` calls `bar()`? It's easier to reason about when the `context.Context` is created at the point of calling `bar()`.
> I would hate having context be inherited from the parent caller.
By default. All I’m arguing is that canceling is in 99% what should be done. You could override that when it makes sense. If your parent wants you to terminate, then in what instances would you keep going? There are some use cases of graceful teardown, but I haven’t seen a single 3p developer give a shit about that, and I’d rather have them respect my desire to yield control back to me, than to go on forever because they forgot to carefully litter their codebase with SetDeadline in their IO calls.
> Do you really want to have to read all possible call-sites to ensure that the context is what is expected when `foo()` calls `bar()`?
Why would I? Unless foo or bar is a unicorn that requires the caller to prepare a custom context, it would work the same as mindlessly passing the context from parent to child, which people do today.
Take the converse example: if today I add a call to bar (no context param), I need to make sure it doesn’t block forever, and if it does, it makes my entire call tree non-cancelable, even if I have been diligent about it in every other place. It only takes one non-cooperative player to destroy that property, and the default is to not be cooperative.
Context isn't a control flow property. Goroutines are still concurrent regardless of whether Context is passed, a global, or whatever, and regardless of whether intermediate functions know anything about Context--they could be calling closures that have captured Context on their own.
Another way of looking at it is that Context is just a way to specify certain properties of messaging objects, not the functions you call. A Context timeout could just as well be set on a socket itself, for example. Await/async is an independent property of the fundamental behavior of each and every function; and a property all functions in a call chain must obey to achieve the desired result.
The "colored functions" debate is just a twist on older debates, such as debates over first-class functions. The definition of first-class functions is instructive. In Go all functions are first-class because, just as in any other language with first-class functions, a reference to a function has the same primitive type regardless of whether it's a closure, is suspendable, etc. The type of its application-defined arguments are irrelevant in this regard. Rust, by contrast, does not have first-class functions in the strict sense, because it has several distinct primitive types for functions; not just for async/await, but notably for closures.
Notably, any language with both first-class functions and closures must be garbage collected. I'm sure there were multiple motivations for Go being garbage collected, but supporting closures as first-class functions is surely one of them, precisely so you don't have a "colored function" problem, where closures are distinguishable from non-closing functions with the same argument and return types.
You can dispute my description and definition of first-class functions (indeed, I skirted the question of whether the distinction between Go functions and methods matters), but at the very least it should elucidate the fundamental issues. Importantly, changes in the type signature of a function because of application-defined argument or return types is irrelevant. What's relevant is whether--or at least the extent to which--internal properties of a function object (e.g. references closed-over values) are visible in the type system, effecting how and when they can substitute for an otherwise identical function lacking the internal property.
The comparison I was trying to draw between them was the way they infect the call tree and usually demarcate impure functions.
When I said that context.Background() discharges the constraint I meant in the sense that the caller doesn’t need to propagate that argument to its caller as it can just pull a context out of thin air.
However I’ve seen a lot of new go devs confuse context.Background() for forking things into the background.
For what it’s worth, I think coloured functions are actually a good thing. If you’re following clean architecture or something similar coloured functions help you easily distinguish what layer in your application something belongs to.
This is not true, and Rust is a counter-example.
Also, that seems like a very excessive build time, either there is some badly configured build script or you are again comparing a full fledged spring app with another vanilla framework.
I was thinking that the GC shapes used in Go monomorphization were related to the structural typing, but thinking about it more, this isn't really the case.
It's not structural typing, but how you use it. One of the biggest benefits of structural typing is how you can have most of the ergonomics of untyped languages while actually getting the performance and safety of sound typing.
Of course writing async code in Go is easier because of goroutines, but apart from that technically they are so different that it is a waste comparing the two.
Unfortunately, the borrow checker forces two extra constraints which often have little to do with the situation:
- No shared mutability. This is an unrealistic constraint; our world calls for shared mutability all the time. Any time a Go reference becomes an index in Rust, you're experiencing this mismatch.
- Single ownership (in the C++ and Rust sense). Most things can be phrased in terms of single ownership, but not all things should be, and it has a cost.
Rust offers you tools to enforce those constraints, but those constraints are often artificial complexity and shouldn't have been added to a situation to begin with. Same with async/await. They solve a self-imposed problem that e.g. goroutines show us don't need to exist in the first place.
We like these mechanisms because they make the program faster, but most of a program doesn't need to be fast. Only the hot path needs to be optimized, the rest of the program should be simpler, more flexible, more decoupled, and have less constraints.
(This is one of the reasons some domains should use Rc and RefCell more and our community should stop vilifying them: they help provide flexibility to balance the borrow checker's constraints.)
That said, this problem really only arises when one tries to use Rust in a domain it doesn't fit well. There are some domains which do fit the borrow checker's constraints really well, and they become benefits rather than sources of artificial complexity.
Isn't the ease with which a reference can be replaced with an index evidence that you usually don't actually need shared mutability?
> For example, in Rust, we often have to use an index or an ID where a regular reference would do in Go. We need to do extra refactoring of function signatures to pass collections down the call stack so we can then use that index/ID... and then find we can't modify a function signature because it's a trait override, and resort to tricky workarounds. In Go, we just use a reference. That choice is decoupled from memory concerns.
The problem is that you can't just dereference an index, you instead need all your callers' callers' signatures to take in the collection, causing a minor refactor shock wave in some cases. If that runs into an unchangeable signature (like a trait override, or a public API), it's pretty much game over and you're back to the drawing board.
If you're making CLI program or small server, this might not hurt that much. In larger programs, the extra refactoring from this constraint can be quite costly and disruptive.
Or is this a lifetimes issue, where the lifetime of the individual reference might be longer than the lifetime of the collection as a whole? In that case, it might be that the lifetime annotations aren't correct, or that the code is wrong in the first place.
I get the idea that you can't change public APIs and function signatures, but that's true in pretty much every language.
Hard disagree.
It is impossible to safely use shared mutable state b/w parallel processes.
See how DB isolation levels work - serializable is the only safe way to go if your processes do anything conditional on the current state. To parallelize, you have to partition the DB which is kind of a cheap way to split one physical DB into separate logical DBs.
So even DBs - the largest shared mutable states we have - are also not really shared mutable states. They need to be broken down into unshared mutable states to be useful.
Shared mutable state is a smell both at the code level and system level.
Also, you can still apply mutability restrictions at the region/thread/process level. You don't need to apply it to every single object, which would lead to the drawbacks discussed above. If you want to learn more about it, take a look at what Pony is doing.
[1]: https://corecursive.com/024-software-as-a-reflection-of-valu...
The problem is that so much of the time when you are building something you don't really have a clear idea of what you are doing, but you build and understanding as you experiment and iterate.
I do writing professionally now and have much the same experience. It is hard to plan exactly what you will write in detail. So much of the greatest ideas materialize as you write. Both writing and coding is IMHO a thinking process.
On the other hand I fully accept that us developers are all different in how our brains work. But I have seen when working with people much smarter than me how much they get stuff wrong and waste time by trying to excessively plan before fully understanding the problem. Stuff I notice I solve easily by taking an experimental and iterative approach.
One small concession: I think JavaScript is awful and Ruby projects tend to end up as a mess. I am mostly a Julia, Lua and Go fan. So I kind of learn towards languages which are a bit in between dynamic and static.
I have worked on one of the largest Ruby monoliths around and I love Ruby, but I also appreciate that Rust can cut through some class of problems that you can't with Ruby (similarly it is much easier to bend Ruby to your will than Rust).
> Go is faster than Java / C#, more memory-efficient than Java / C#
This is probably the worst part of it, because it doesn't substantiate any of these claims. Look up any set of benchmarks online and you'll find this claim to be mixed at best. Suffice to say if performance actually matters for your app, you'll squeeze plenty out of C# and Java just as you can with Go. And all three languages have no shortage of pitfalls for performance if you're not careful.
Go is just a different language. And the author liked it more when they wrote this in 2019.
> Go is simple so that all of this can hold true when confronting the average Go program with the average Java / C# program. It doesn’t matter whether Go is truly faster than C# or Java in an absolute sense. The average Java / C# application will be very different than the best theoretical program, and the amount of foot guns in those languages is huge compared to Go.
Not to say that I knew perfectly well where Go sat vs Java vs C# in terms of raw performance, but mine was a different point based on my experience.
>Look up any set of benchmarks online and you'll find this claim to be mixed at best.
I don't know a ton about low level performance, but I'd find it surprising if it's not true. How can interpreted bytecode outperform direct machine instructions?
The benchmarks I've found have Go outperforming Java and C# in almost every test.
https://benchmarksgame-team.pages.debian.net/benchmarksgame/...
No production Java/.NET runtime directly interprets bytecode, it gets dynamically compiled into machine code. And as a result it can dynamically _recompile_ it if it discovers runtime profiling patterns that mean a different compilation can run faster, it can dynamically inline functions into the compilation if pertinent, etc.
This does not mean that Java/.NET code _will_ always run faster than native static compilation (a quick gander at the real world shows that optimized production code is usually pretty close either way), but it explains how it _can_ run faster.
Oh, didn't know that. Interesting, thanks!
C# actually does really well on this front in that it brings in value types front-and-center, although similar types of capabilities exist in Java(either through ByteBuffers or sun.misc.Unsafe) if a bit harder to use.
True, with the caveat that:
1. Compilation occurs while the program runs, using some CPU power
2. Compilation occurs on those parts of the code that pass certain criteria for being a hotspot
3. Because of #2, some CPU power has to be used for profiling continuously as the program runs
It can be faster than AoT compilation, but AoT can do much more aggressive optimisations because AoT compilation can use all available CPUs, for as much time as they want to. JiT compilers have to balance the processing power used for compilation against leaving some processing power for the actual program.
JiT performs very well on benchmarks because:
1. It's a small piece of code, run serially (thereby leaving one entire other core just for profiling and compilation)
2. That one small piece of code is run dozens of thousands of times, triggering the JiT compiler to optimise that "hot spot".
3. The "hot spot" is the only code to run so the entire program is very quickly turned into a native-code program, with the best optimisations that the JiT compiler can perform.
In practice (i.e. not benchmarking), the code is large, it doesn't run serially, it uses all cores (especially in performance sensitive applications) and the JiT compiler and continuous monitoring will effectively steal processing power from the program. In benchmarks, the JiT compiler is not using any power that the program would have used.
> And as a result it can dynamically _recompile_ it if it discovers runtime profiling patterns that mean a different compilation can run faster, it can dynamically inline functions into the compilation if pertinent, etc.
There's a lot of "if"s there. They all have to line up perfectly to get well-optimised native code.
> This does not mean that Java/.NET code _will_ always run faster than native static compilation (a quick gander at the real world shows that optimized production code is usually pretty close either way), but it explains how it _can_ run faster.
It can run faster, but that is rare outside of the benchmark environment - it's more likely to run as fast as AoT compiled programs, because the throttling factor in most programs is the data access patterns, not the computation.
In practice, for the usual type of program, you're not likely to notice much of a difference (other than startup time) between AoT programs and JiT programs.
All Java implementation have flags to JIT without interpretation.
Their major implementations allow for PGO sharing across runs.
Finally, both ecosystems have supported forms of AOT for the last 20 years.
Without opining on whether it does, it definitely can, because the Java JIT compiler has access to runtime information unavailable to the go compiler - which enables more aggressive optimisation on hot paths.
Because the JVM and .NET compile; they don't interpret bytecode.
They will almost always be slower because they have 3 structural disadvantages.
1. Compilation bytecode => machine code happens at runtime so it's a constant overhead. There's also other overhead related to keeping invocation counts to decide when JIT should kick in.
2. Because compilation happens at runtime, they can't spend too much time on compilation so they are limited at how good the generated code can be. Go doesn't have those restrictions.
3. Modern performance is heavily influenced by memory use. By design JVM and .NET are more memory hungry. Every object in JVM has 8-16 bytes overhead compared to Go struct etc. Value types are relatively recent and not commonly used. In practice it adds up.
The are 2 advantages of advanced JITs:
1. It can make better inlining decisions based on actual execution counts of a function vs. a heuristic in a Go compiler. (Go 1.20 will have preview of profile-guided optimization which provides the same optimization).
2. A generational GC used in most JVM / .NET implementation is faster for certain allocation patterns (high rate of allocations / frees) than Go's allocator.
But your average Go program will be both faster and significantly less memory hungry than your average Java / .NET program.
The memory overhead is especially bad for Java / .NET. Go only compiles the functionality used.
Java / .NET have very heavy runtimes / standard libraries that are packaged into one library and have to be loaded into memory.
What it means in practice is: I can run a Go server comfortably on the cheapest shared server with 512 MB and a Java / .NET server would push it to the limit.
Value types in .NET exist since 2001.
.NET has unsafe since 2001.
.NET has had AOT since 2001 with NGEN.
C++ is one of the supported languages on .NET since 2001.
Please educate yourself on the platforms, before criticising them.
- JIT and runtime optimizations
- AOT compilation if you want it (probably contends Go's native compiled code?)
- Access to raw pointers that lets you do things like pinning things in memory and intervening with normal GC
- hardware intrinsics for SIMD support
C# comes with some serious performance knobs if you're really committed to writing performant code.
Currently compliant with C++17.
The C# measurements from the same source don't seem to show that?
https://benchmarksgame-team.pages.debian.net/benchmarksgame/...
I really dislike this type of argument. It isn't the language itself that is better, it is the community.
This article should just be called "I like Go better than other languages". It barely even discusses Rust or any of the truly unique things that Rust actually brings to the table vs the GC big 3 (Java, C# and Go).
The reason inheritance is not popular in Go is that it's not possible.
The reason IOC containers are popular in Java and not Go is that Java makes them relatively easy and Go doesn't.
Go is drastically less complicated language than Java, it has significantly less foot guns.
As a result Go community self-selects to people who value simplicity and as a result you do get a different community.
But community is shaped by the language so it really is about the language.
Is it? Java is a very bare-bones language, remarkably so given that it is 25 years old.
On the contrary, most the Javascript I wrote in 2015 is essentially defunct. I've developed a deep dread for starting anything new in it.
I've noticed over time that the various linters and vet-tools have become more strict about things that they didn't detect in the past.
But I tend to write code with reasonably high levels of coverage (in the 80-100% region) so running linters and test-cases has always been part of my approach to using go.
Having programs run 7 years later should be the expectation, not the unexpected.
What boggled me at the time was how JS programmers had no idea that the current at the time async programming in JS was a war zone - they had no other experiences and took it as normal.
I struggled to understand what I was missing, and then I happened to find this post, and it hit pretty much every point I encountered in practice:
Why a pizza and not a restaurant meal?
Why a scooter and not a skateboard?
A different set of tradeoffs. Sometimes one wants a limited but easy thing instead of a powerful thing that requires significant mastery to handle.
Both Go and Rust and general purpose languages. You can write most (but not all) kinds of software in both of them so a claim that Go is more "limited" than Go is bs.
As to "mastery": I've spent significant amount of time mastering C++ and got pretty good at it.
Go was both significantly easier to master and I'm significantly more productive in Go.
So what exactly is the justification for C++ (and Rust) complexity?
Doing it the hard way: anything with a complex domain e.g geospatial applications where you have an exponential explosion of different logics for different types of shapes or modelling medical ontologies.
You can write anything in any language the issue is if you're taking the path of least resistance to do it and how much risk you're exposing yourself too with variable skill levels of developers doing it.
E.g see all the frankenpython implementations kicking around banks and Instagram to use the "simple" language that turns into something much more complex and brittle.
Name a program you can write in <language> that I can't write in <language> is a terrible way to compare languages. That applies for any Turing complete language, but I wouldn't write a game in emacs lisp.
> Go is more "limited" than Go
Assuming you mean "go is more 'limited' than rust, you can write all software in both languages. I'm not sure who claimed it either because both the linked article and comment above you don't.
The justification for C++/Rust is largely the same as the justification for any lower level language: performance. Theres a lot of cases where GC pauses are a no go (pun intended). Well written rust is going to be more performant than well written go. You have greater control over threads, for better or for worse.
The benefit of rust in particular is the approach to memory management, package management, and how hard rust it is to shoot yourself in the foot. Rust filled a gap where people wanted a performant, expressive low level language without the memory hassle that came with c/c++.
Why Go and Not Rust? - https://news.ycombinator.com/item?id=20983922 - September 2019 (477 comments)
Go is simply more productive to write. Now, for things where performance or memory really matters, rust is beautiful and wonderful (and the language is very fun!). These use cases, like blockchain or specific algorithmic stuff, make rust a compelling choice - the new C++. But for rolling up your sleeves and getting stuff done quickly (aka shipping fast), go is an absolutely fantastic choice. Go seems to be as much as 50% faster to ship features based upon my unscientific observations.
For what it’s worth, the only two languages our company uses for backend services are go and rust, so I love both languages (otherwise we’d be using other languages). I find rust to be particularly fun and interesting to code in, but for most microservices, go reigns supreme. It’s simple, readable, and nice generally (though error handling could use some work). For companies with a mature feature set, I’d imagine rust is highly preferable.
This is not generally supported by benchmarks that I have seen (random link below).
https://benchmarksgame-team.pages.debian.net/benchmarksgame/...
I looked at first benchmark (funkuch-redux) where C# significantly beats Go (8.3 vs 32).
It turns out that the winning C# programs uses SSE2 intrinsics.
The second best C# programs virtually ties with Go implementation, which is not a surprise: those are pretty straightforward, calculation heavy programs. Easy to optimize so both .NET JIT and Go compiler do similarly good job generating code.
So all we've learned that using SSE2 can significantly speed up certain kinds of code but it has nothing to do with C# vs. Go.
Arguably SSE2 intrinsics are actually part of C#. Apparently they were added as a library in .net core 3 (in 2019?).
You will typically not use intrinsics in your code because a) you don't need them b) they're pain to use
And I could write a Go version that uses intrinsics and it would be equally fast.
The same applies to n-body and spectral-norm at which point I've lost interest in those benchmark because all they show is that someone spent extraordinary amount of time optimizing C# code with intrinsics while the same effort wasn't applied to Go programs.
But by constructions you average Go program will be faster that your average .NET JITted program.
JIT has constant overhead and can't spend as much time generating best machine code as offline Go compiler.
Plus .NET is more memory hungry (.NET object have memory overhead not present in Go structs), UTF-16 strings take up ~2x more spaces than Go's byte strings etc.
Doesn't mean every Go program will be faster than equivalent C# program but it's more 80% in favor of Go.
Here is another showing Go under-performing both Java and C# by a bit. Benchmarks may not be perfect but better than conjecture.
Also, one of the least toy benchmark out of these (binary tree) which basically just stress tests the GC shows Java and C# multiple times faster than Go. And while you may claim that “you can just use value types” with Go, GC will be a significant part of any practical application and Go is absolutely beaten in this category by quite a huge margin.
https://uptrace.dev/blog/posts/go-memory-arena.html#memory-a...
Intrisics are part of the standard library, it isn't .NET fault that Go designers don't want to provide similar facilities.
"After all, facts are facts, and although we may quote one to another with a chuckle the words of the Wise Statesman, 'Lies--damned lies--and statistics,' still there are some easy figures the simplest must understand, and the astutest cannot wriggle out of."
Leonard Henry Courtney, 1895
> So all we've learned that using SSE2…
As you say, we've learned that a C# program not using SSE2 was a little faster but more-or-less the same as the fastest Go program shown.
> JIT has constant overhead and…
<PublishAot>true</PublishAot>
Maybe that is a stupid reason but I stand by it.
I never worked at Google and presumably I don't have Google's problems and yet Go is the best language I found to wrote those programs. Simple, fast and productive.
Could you connect the dots for me: what exactly is that Google needs that I don't and how does that map to design of Go?
Is it compiled because only Google needs statically compiled, easily cross-compilable, easy to distribute executables?
Is it portable to almost every OS / arch available because that's something only Google needs?
Is it statically typed because only Google cares about catching type errors at compilation time?
Is it garbage collected because only Google cares about the productivity and memory safety of programs?
Which Go design features solve problems that only Google has?
Sadly, I think I cannot use Rust to the fullest since the RLS (language server) is as slow as the compiler itself, and you find yourself waiting quite often to realize you made a mistake a couple of lines before the one you're typing. Let alone guessing the types and stuff because the compiler / RLS didn't catch up to what you wrote so far.
I hope the speed of the Rust compiler / RLS will improve soon, it has been like this for the last 4 years :(
Isn't RLS deprecated for rust-analyzer already for some time?
Instead, I'm going to lament about a few things I think would take Go from an extremely good language, to one of the best languages out there:
* Context. Context-creep is really annoying; similar to async in JS, it becomes something that over time absorbs your codebase and dirties function signatures beyond recognition [1]. Unlike promises in JS; for surprisingly low benefit. I don't for sure know what a better solution looks like, but my initial thought: the language runtime should have globals like setContext() and getContext(), which can set and get a goroutine-local context. If you want to pass a context between goroutines, just do it like you'd do it today.
* Enums. I will die on this hill: proper enums (think typescript and rust) are one of the most powerful programming paradigms in typed languages. Go not having them is one of its largest gaps; and leads to really weird obviously-should-be-an-enum things that instead get shoved into a set of constants with a common prefix [2]. What do you lose out on? So. Much. Beyond the obvious [3]; one of the coolest features of enum-prioritizing languages like Rust is exhaustive switch/casing; this is a huge feature in application development, and that's what Go is great at; except for this gap.
* Pointer-to-interface shenanigans, and how that oftentimes interacts with custom error handling. This is the WILDEST part of Go, and I am convinced that even extremely seasoned Go devs run into this, know that it does happen, try to avoid it, but can't explain why it happens. "Oh yeah, uh, nil, when coalesced into an interface, is nil, but, uh, also isn't, uh, nil, uh, when it comes to, uh, checking if its nil" [4] WHAT. There's no excuse for this; this isn't a feature, this is a bug, end of story. Its a bug that, for some reason, has lasted years; in the same breath that defenders explain why it happens they also say "oh, but, Go is industrial scale, its for large engineering teams"; this is not industrial, this is brittle.
[1] https://pkg.go.dev/github.com/aws/aws-sdk-go-v2/service/s3
[2] https://pkg.go.dev/net/http#pkg-constants
In the vast majority of all Java programs that I have written, there is typically a single try/catch block at the very top level (and occasionally in low level IO code) - and none of the intermediate layers of application logic need to concern themselves with error handling (exception handling) at all. This is as it should be.
The needless ceremony required to check and handle errors on every function call sounds tedious and I can’t see how it contributes to clarity. Try/catch is a blessing.
Rust achieves a similar degree of ergonomics with its `Result` type and `?` operator. You can propagate the failure case of functions that return a result easily with `f()?;`. While it does require ‘annotating’ these functions by declaring them to return a `Result`, unlike Java, that explicitness fits well into Rust, and is trivial to compose using `?`.
Instead of multiple if/then or match blocks, you can write `let x = foo()?.bar()?.baz()?` or similar. This also works with Rust’s `Option` type, and can be implemented for arbitrary types (as I understand it) using the `Try` trait.
Simple monadic programming without the complexity present in other functional languages.
And I haven’t even touched on the benefits of Rust macros. TDoes Go have anything like Rust’s `dbg!`, or its statically-typed `println!`, `format!`, etc.?
Agreed about the Pointer/interface/error shenanigans though.
Yet having a compiler be able to tell you « this code isn’t handling all the cases » is really really valuable..
Edit: about the error interface nil casting, i had this problem just 2 days ago, fortunately the linter detected the problem (not the compiler) with a weird « this code is never reached » error. I would have never guessed this behavior otherwise.
This isn't okay! Like, the other stuff I complain about or ask for improvements is whatever, but this is something that if any Go devs are reading, legitimately needs to be fixed, even if it means a language breaking change. There's no logical or reasonable explanation as to why that conditional on line 33 passes. There's definitely a reason; I don't care what it is, its a bad one.
Its a surprisingly common situation to run into once you start hitting medium-sized codebases and custom errors; where you have some module returning a custom error, some other module returning its own custom error or just an error, and you need to coalesce them back into just a plain-old-error. The SOP at my 1000 engineer big-tech 90%-go-shop is to literally never specify custom error types in a function's return signature. Its too unsafe; you know, that thing that's supposed to make your code more safe, providing more information to the type system, yup it makes your code less safe. Return the custom error (concretely), specify 'error' in the signature, and put in a comment that it can return that error type for eventual type coercion.
How embarrassing is that for the language? Reminder, team: its 2023.
type A enum (A, B, C)
Ok I'm started. Iota is a solution in search of a problem; a solution which was designed to replace far better solutions (enums) worse, and ended up solving practically no problems.
Its issue is really obvious once you think about it and what enums are oftentimes used for: data serialization. In the face of a constantly changing code-base, iota doesn't make any guarantees that some enum key will always be equal to the value iota increments out. Today, UserStatusDisabled is equal to 1, but tomorrow it could be equal to 2 if someone adds other iotas around the code; and detecting this statically is difficult, because there are (very, very few, but present) valid use-cases of iota, and its impact on the resulting value depends on the order the statements are evaluated.
This makes it functionally useless in any situation where an enum's value is serialized external to the application; e.g. in JSON structures returned to clients, messages published to Kafka, or database schemas serialized to SQL, which is a pretty substantial number of places where enums in general are useful, and also a list of things that seem pretty damn core to what a "systems programming language" would be used to do!
Additionally, an integer contains no context; so an enum like "UserStatusDisabled = "disabled"" is an awesome thing to serialize in JSON to a client; but "UserStatusDisabled = 2" is a horrible, horrible thing. Obviously, technologies like gRPC functionally do act like this; but the schemas are a shared contract. Go variables aren't shared, and they aren't a contract.
Which is all to say that my company also has a static linter that rejects any usage of the iota keyword. Go is just littered with mistakes like this.
I don't do as much Java as I used to, I mostly code Python on the backend, but if speed/scaling becomes an issue then Java could be a good alternative to Go for enterprise software, despite being considered "boring" by many.
I generally think a garbage collector makes sense for 99% of programs out there, but databases often need a bit more control over memory management for performance reasons.
There are many parts to a distributed database, and there is surprisingly more business logic than you might expect. The business logic is mostly what I am talking about - being able to develop quickly in Go for these less performance sensitive parts is a huge boon.
For performance sensitive parts I think there's an argument that we could have been happier in a language like Rust, for example a dual process model like TiDB's. But I think it's ambiguous: a more complex architecture like that one comes with huge tradeoffs.
tl;dr there's much more to database development than performance at all costs, and it's false that the team regrets using Go, please don't spread that rumor :)
And nowadays there’s even more separation between the two.
I stand corrected.
> And nowadays there’s even more separation between the two.
That is almost entirely irrelevant. A project’s culture is about 90% the culture of its initial group. Even if Go(Lang) would be transferred and entirely sponsored by, say, the EFF or something, I would still not trust the language to be a good fit for the wider community.
Eh, not in the case of Rust. Rust doesn't even use the MPL.
The biggest Mozilla-ism I can think of that remains in the Rust project is, like, the use of "r+" or "r=me" for code reviews.
sorted