Hacker News Reader

That's about right in aggregate, though some of the specifics go either way. I have two theories why so many people are convinced that Go is much lower level than it actually is, and I'm not sure which one is true:

a) Go was really sold as a "systems programming language" early on. Maybe some adopters never really questioned what that means or if that's true?

b) Go produces actual executables, with what in Java would be the JIT executed ahead of time and any runtime bits statically linked in. That is completely orthogonal to abstraction level, of course. But maybe the fact the there's no externally visible runtime makes people group the language into a bucket with C?

There's a couple of things that make Go feel lower level than Java. The first is having access to pointers and the manipulation that brings. Even though there isn't pointer arithmetic, needing to think about referencing and dereferencing structs is lower level than Java, which doesn't provide this in the language at all and forces all objects to be pointer-heavy references of references (apart from primitives).

The other thing is that by not having a VM, a lot of usual Go performance optimizations end up being "closer to the metal." E.g. optimizing struct alignment is a fairly pedestrian Go performance optimization. On the other hand, if you rely on guarantees of how the Hotspot JVM (as opposed to other JVMs) lays things out in memory, that's considered pretty deep hacking in Java.

If these are really the two sticking points, then Go should be on roughly the same abstraction level as C#/.NET (which also has pointers - with arithmetic, even! - and user-defined value types with explicit layout control and unions).

However, given that Go also brings its green threading into the picture, I'd argue that it's actually higher level than either Java or .NET in that one respect.

C# is an interesting case. You can only use pointers in unsafe C# code, so you could argue that unsafe C# code is as low-level as Go or even lower (which I actually think you'll find many supporters for), but unsafe C# is different from normal C# and is outside the purview of everyday C# code (whereas again pointers in Go are absolutely pedestrian).

RE threading, green threads though are coming (back!) to Java as well with Loom, but I consider that more of a library/runtime thing than a language thing (as is the case with Loom). A hypothetical Go library could expose system threads and have an API for manipulating them, it just wouldn't be first-class in the same way goroutines are. But this is similar to how having async-await in C# doesn't negate the fact that you can use direct system threads from the standard library.

With C# and .NET, you also need to consider the different kinds of pointers involved. The thing that's closest to Go's pointers would be managed references (which don't have pointer arithmetic but are GC-aware). In C#, this corresponds to in/ref/out parameters, locals, and return types. Now, the difference is that those can point directly to the stack, and thus cannot escape the function in which they were created for memory safety reasons. But conversely, they are allowed in safe code.

And then there's stuff like Span, which these days lets you do something similar to Go slices:

   Span<byte> p = stackalloc byte[n];

(note that this is also safe code!)

As for green threads, I think in case of Go they can't really be considered a library thing due to language integration of goroutines and channels. It's also not quite the same as C# async/await, because the latter isn't really a threading system so much so as syntactic sugar for CPS; it's entirely possible to have a single thread running the event loop.

refs (and their various flavors) aren't really pointers though. They're a higher-level abstraction on top of pointers. For example, you can't have a ref of a ref, nor can you use a ref as an independent type, whereas both those things are possible in Go. refs, as their name suggests, are basically higher-level annotations to allow for pass by value or pass by reference, whereas pointers are lower-level independent types (usually in languages which only support pass by value). The point being here that references are a higher-level concept than pointers since you usually will use the latter to implement the former.

> As for green threads, I think in case of Go they can't really be considered a library thing due to language integration of goroutines and channels. It's also not quite the same as C# async/await, because the latter isn't really a threading system so much so as syntactic sugar for CPS; it's entirely possible to have a single thread running the event loop.

Again this isn't really a language thing anymore though, it's more of a runtime/operational detail. You could implement goroutines with CPS if you wanted and async-await could be implemented by green threads. Off the top of my head I can't think of anything in the API of goroutines or async-await that would allow you to distinguish whether they're being implemented by CPS or green threads in the background.

> refs (and their various flavors) aren't really pointers though. They're a higher-level abstraction on top of pointers.

This is not entirely true. Refs on lower level are pointers - when it compiles down to CIL, the resulting types are rendered as T&, and referred to as managed pointer, as opposed to T* the unmanaged pointer.

And you can use T& as an independent type - it's not a high-level annotation on a pointer, but a fundamental CIL concept. The limitations that you describe are all about not allowing a T& that points to a local on the stack escape the function where said local is defined (whereas Go silently lifts the local to heap in this case). Hence why you can't have T&&, and why a struct that has a T& field must itself be declared as a "ref struct" (and have the same restrictions applied to it).

On C# level, refs were originally a function argument modifier, as you say, solely about pass-by-value vs pass-by-reference, implemented under the hood as managed pointers. But since then, we've got the ability to return refs, declare local ref variables and fields (in ref structs), and reassign them. So at this point they're effectively types, with some restrictions on use similar to Rust borrowing.

> You could implement goroutines with CPS if you wanted and async-await could be implemented by green threads. Off the top of my head I can't think of anything in the API of goroutines or async-await that would allow you to distinguish whether they're being implemented by CPS or green threads in the background.

To implement goroutines with CPS, you'd need to rewrite every call into CPS-style. This is technically not observable, yes; but perf would be atrocious in practice, so it's not going to happen.

Async/await, though, is literally defined in terms of callbacks in the language spec; threads are completely orthogonal to all that. The dispatch loop can decide to run continuations however it sees fit, including threads - but at this point we're talking about callback-oriented code that is already desugared from "await". And running that on top of green threads is possible but pointless, since the semantics of await requires that async context switching happens only at awaits; but the ability to pre-empt at any arbitrary point is precisely the advantage of green threads.

I never got around to replying to this, but on the off-chance you're still following this thread (and any other readers still reading)

> This is not entirely true. Refs on lower level are pointers - when it compiles down to CIL, the resulting types are rendered as T&, and referred to as managed pointer, as opposed to T* the unmanaged pointer.

Right and I don't think anybody would disagree that CIL is not a low-level language. But plenty of high-level language features compile down to just pointers. This doesn't mean that the language offers pointers though! In that world I could say that Python offers pointers (or indeed any high-level language that compiles to LLVM and has a feature that just corresponds to a pointer).

> But since then, we've got the ability to return refs, declare local ref variables and fields (in ref structs), and reassign them. So at this point they're effectively types, with some restrictions on use similar to Rust borrowing.

No they're not. You still have the `Ref<...>` vs `ref` distinction (which is why ref fields aren't a thing), you still can't have nested `ref`s, and you still can't switch between "ref"ing and "unref"ing something, i.e. ref-ing is infectious.

Likewise the fact that refs have some linear characteristics doesn't make them pointers (in the same way that Linear Haskell doesn't mean that Haskell now has pointers). The reason why we say that Rust has pointers (apart from the raw pointers), is that they don't have any of these issues!

> To implement goroutines with CPS, you'd need to rewrite every call into CPS-style. This is technically not observable, yes; but perf would be atrocious in practice, so it's not going to happen.

Yes there are sound reasons for why Go has chosen certain implementation decisions, but those are not part of the language semantics, which is ultimately what determines whether a language feels high-level or low-level.

> And running that on top of green threads is possible but pointless, since the semantics of await requires that async context switching happens only at awaits; but the ability to pre-empt at any arbitrary point is precisely the advantage of green threads.

No current industrial green thread implementation I know of offers pre-emption, they are AFAIK all cooperative. IIRC Go's implementation yields on channel sends and GHC's implentation yields on memory allocation. I think Loom is heading for pre-emption, but it hasn't been finalized yet. But that's not really the point. My point is special syntax for a concurrency feature that is not backed by system threads does not preclude a language from using system threads, which is the fundamental relationship between Go's goroutines and any FFI that would expose system threads. Or put another way, being unable to use system threads are not a fundamental part of the semantics of Go, while being unable to use pointers are a fundamental part of the semantics of safe C#.

You do have more precise control over the memory layout of your data in Go. Because of the value types and support for using interior pointers to arrays. In that respect it's like C which is lowish level.

You can know your class alignments in Java using tools like http://openjdk.java.net/projects/code-tools/jol/

Having access to pointers is a superficiality. Java is getting value types, and C# already has them.

Right but those tools are VM-specific (e.g. the one you've posted is Hotspot-specific) and VM-specific optimizations are usually considered advanced optimizations in Java.

Valhalla's value types aren't the same thing as pointer access (or lack thererof). The first thing is that this is baked into the definition site rather than being something that can be decided at a call site. The second thing is that value types entail way more than just pointer access (as befits a higher-level approach). In particular it gets rid of the built-in ability of Java objects to act as locks, gets rid of mutation, etc. It's a much larger change than just referencing and dereferencing.

More to the point, the notion of pointers referencing and dereferencing is a lower level concept than reference vs value types. The former is usually used to implement the latter.

> More to the point, the notion of pointers referencing and dereferencing is a lower level concept than reference vs value types. The former is usually used to implement the latter.

If the end result is the same, then the claim is really meaningless.

I'm just pointing out the parroting that goes on in the golang community.

> If the end result is the same, then the claim is really meaningless.

One being used to implement the other doesn't mean the end result is the same (you cannot use value types and reference types to implement pointers because of the call-site vs definition-site distinction). As I mentioned, value types are far more than pointer dereferencing.

The point is, pointer dereferencing in golang is limited to mimicing value types/reference types, something that C# already offers. It is not like C and C++ where you have pointer arithmetic. So it's just an illusion that it is "closer to the metal".

No it's not. Value types and reference types are definition-site concepts whereas pointer referencing and dereferencing are call-site things. This allows both for different ways of programming as well as different language semantics.

> It is not like C and C++ where you have pointer arithmetic.

You absolutely _can_ do pointer arithmetic—it just requires using the "unsafe" package for obvious reasons (https://go.dev/play/p/Batmgb1KJcg).

Hell, there's even a helper function (https://pkg.go.dev/unsafe#Add) for addition!

Calling unsafe is not the point here. You can do similar things in Java if you really need to.

It's also worth distinguishing between "has value types" and "uses value types". C# has them, but they aren't particularly idiomatic--they're regarded as an optimization.

They're reasonably idiomatic. Anyone dealing with the standard library will likely have to deal with KeyValuePair, for example - which is a value type. So are the (language-integrated) tuples.

I take your point--idiomatic C# code uses some value types. It would be more accurate for me to say there's a difference in degree with respect to the extent that C# and Go employ value types: wherever C# uses classes, Go uses structs; where C# uses class instances, Go uses either struct values or pointers to struct values (or interfaces, in the case of runtime polymorphism).

Tuples are a great example that I might not have thought of -- you practically "define" your own value types all the time. Also, all enum types are value types. And there are lots and lots of very commonly used value types in the base class library: DateTime, TimeSpan, Guid, CancellationToken. Programmers have to be familiar with their semantics.

Yeah, this makes sense. I concede that idiomatic C# uses value types sometimes, although they're more pervasive in idiomatic Go.

How is that relevant to the point whether a language is "closer to the metal" or not?

Because calling a language "close to the metal" is potentially ambiguous--it can imply that a language merely has "close to the metal" features or it can imply that they not only exist but are used in the course of idiomatic code. Specifically, only one of these interpreters conveys the idea: "I can read code in this language and have a pretty good idea about what the generated machine code is like".

> I can read code in this language and have a pretty good idea about what the generated machine code is like

This is not true even for C.

Secondly, it seems that its common in the golang community to parrot things that not only are incorrect, but provably wrong.

First of all, you can look at C code and have a pretty good idea about the generated machine code, or at least a much better idea than with C# and Java. Secondly, it's even easier to predict Go code because Go compilers don't optimize as aggressively as C compilers and because Go has very little UB.

> Secondly, it seems that its common in the golang community to parrot things that not only are incorrect, but provably wrong.

Whatever your intentions may have been, this is just language war flame bait, and it's against the rules.

> Java is getting value types,

And it is good thing to have. But if one desperately comment same thing multiple times, I feel reasonable to ask why the hell they weren't added in all these years?

The philosophy behind Java is that a feature needs to be demonstrated in production very well before it is added to the language, in order no to go overboard with features and end up with C++ :)

Value types have proven their worth, especially with the gap between CPUs and memory bandwidth, so they're being added.

Go has a C style error handling that by virtue of being very bad, composes a large share of the code one writes. That pushes people impression of it near the one of C, even though it has some very high-level features.

I agree completely with your reasons. Generally, people just have no idea what is going on.

This is not our experience at all.

Our experience is that typical Java programs will consume a lot more memory, and spend a lot more time doing GC and other unproductive work than typical Go programs.

Go provides a lot more tools that give more bang for the buck when optimizing. For example, everything in Java is a reference to some other distant blob of memory. This does not work well on todays cpus compared to being able to colocate things.

We run more stuff on the same machines than we could with Java.

Team wise, Go is easier to learn, and scales better with teams. We don't have trouble with team members struggling with certain team member's code / abstractions like we did with Java or C++. Switching between the two, there is a lot more cognitive overhead in Java land. It's understandable that this may be boring to some, but it is a great feature of the language.

> and spend a lot more time doing GC and other unproductive work than typical Go programs

That’s definitely way in the other direction - Java’s GCs run circles around Go’s. (Part of) the reason for Java’s bigger memory consumption is that it is very lazy when it comes to GC. Go runs it all the time for lower latency (at the price of much lower throughput) while Java allows for control it.

Java GC has to be very good considering Java could take 10 times more memory for typical data structures compared to memory efficient layouts.

Not sure if sarcastic, but that has nothing to do with the quality of a GC algorithm. Sure, Java does not (yet) have value types so data structures with non-primitive elements will not have the most optimal layout, but java does objectively have the state of the art GC implementations that work under so high allocation rates that other GCs simply break down. Eg. Go notably slows down threads to let the GC catch up with the work load.

Go is "lower level" in the sense that it depends much less on the JVM to optimize things for you. Go code maps much more neatly and predictably to machine code because the optimizations are fewer and less aggressive (e.g., Go has semantics for memory layout). This doesn't imply that Go is inherently faster than Java, but it's easier to reason about the performance of Go code.

It's slower or faster but certainly not overall slower.

By default Go will be faster because of the difference of value types between the two languages, Go is also easier to optimize since the language is simpler and has less abstraction.

Go can be much, much faster if fast compile times is not focus. A good compiler can probably do some compile-time RAII inlining like C++ and Rust and reduce GC usage without needing any major changes to the core Go language. A lot of the more advanced optimizations done by LLVM are not implemented in Go because they slow down compilation. Go compiles insanely quick (like classic Pascal/VB), it feels more like a JITed language than compiled.

Does RAII make code faster? I thought RAII just meant things were freed as soon as they went out of scope rather than waiting for the next GC sweep? Seems like this would make code slower (instead of one free() for all garbage, you have to do a free() for each allocation). Granted, the tracing burden is lower, but I'm of the impression that tracing is cheap?

> Go compiles insanely quick

Compared to Java?

He's implying that it's comparable to Java in that it compiles fast:

> it feels more like a JITed language than compiled

Yeah, and one need to compare `go build` with `mvn clean compile/package` not javac. Because Java does not come with inbuilt tools to compile whole projects.

Too bad Java doesn't have an option to disable all runtime type reflection, etc... it would be very similar to Go, I bet. Especially if AOT compiling was used.