Hacker News Reader

Comment by neonsunset | Hacker News Reader

[flagged]

Comments extolling the virtues of LISP or the CLR in a Go thread are "not negative about Go"; they're simply off-topic.

An on-topic, comment critical of Go would be: it's silly that cgo is considered so bad (why?) that people go to such great lengths to avoid it, when so many (otherwise great) Wasm runtimes already exist.

But if I'm already using Go and need a Wasm runtime, my options are wazero or wasmtime-go (etc), not LISP and C#.

I would argue that attempting to hijack a random Go thread to complain about CGo is also off-topic. They can write a coherent article and post that, if they want, but repetitively distracting from the actual thing being discussed is tiring.

Also, CGo design is a trade-off, and makes perfect sense. Nobody has come up with a design to have both fast C ABI FFI (no stack/thread switch) and cheap green threads (small stacks). People complaining about CGo rarely seem to understand this.

cgo is extremely slow, particularly in programs with many active goroutines.

I regularly look sad at profiles of production cgo programs where findRunnable is in the top 10.

Right, and both modernc and wazero are two highly sophisticated projects being used to avoid it.

But cgo being slow (and above all displeasent to work with) is what I'd consider valid criticism of Go.

Go has other advantages, FFI is not one of them.

Can you toss a reference to modernc? I can't seem to find the project.

Most projects live in the gitlab below; modernc is the vanity import which alone doesn't resolve to anything. The most famous project is an SQLite driver.

https://gitlab.com/cznic

https://modern.org/sqlite

I thought Go could handle a million-plus active goroutines? Are you saying it'll work but be poor performance compared to something equivalent in C++, JavaScript, or Rust?

A quick search reveals cgo is mainly slow due to overhead when invoking non-pure go functions (i.e. C-functions).

https://stackoverflow.com/questions/28272285/why-cgos-perfor...

How does this relate and interact with goroutines in a performance impacting way? As in, why is the case with goroutines any different?

Edit: @neonsunset: Thanks <3. The thread you linked covers erl/BEAM and C#/dotnet, and makes salient points of skepticism about the practical need for millions of routines. I'm still curious what makes the Go story worse or different?

Go can "handle" a million plus active goroutines, but that doesn't mean Go can handle a million goroutines doing literally anything a goroutine can do. They still have to fit into the resources of the computer. They can not all, for instance, have a gigabyte of their own active memory that they are doing something with, even though Go and a goroutine can individually "handle" a gigabyte of active working set.

Cgo calls take a lot of resources, or, at least, take a lot of resources if you're trying to do a million of them simultaneously. (In absolute terms, Cgo slowdown is one of those things that programmers hear bandied about in conversations and can easily come away with the impression that a single Cgo call will consume 150 milliseconds or something and instantly bring your program to a crawl, but a single call is not noticeable. It's only really a problem when scaled somehow. See also the belief that GC guarantees that my program is going to be frozen for 250 milliseconds every couple of seconds.) In this context probably the most relevant one is they need a full OS thread to work in the C runtime, so a million simultaneous Cgo calls would require a million simultaneous OS threads, and that's still a pretty big expense today.

For obvious reasons, Go can only "handle" millions of goroutines if all but some approximation of the number of CPUs available are doing nothing at any given time. The only common type of program that might have this circumstance is really network servers.

Sure you can have a million goroutines, but you’ll now have more scheduler and gc overhead, and this is part of the issue.

Calling out to ffi requires preparing both the stack and the runtime for that call. It’s theoretically possible to avoid the need for this preparation, but with some caveats - you have to block the gc view of parts of the world (the parts ffi could be using) and you need to ensure that your runtime isn’t interacting badly with the ffi environment (e.g. not sending it signals and so on). Go’s arrangement (as is common with most runtimes) needs to active work to prevent both problems, and the cost of cgo is largely the cost of doing those things. Most managed runtime/gc languages have these challenges and it’s one of the axes that people say makes them not systems languages (a poorly defined term for sure, but fair on an axis of “can’t make calls to other software modules at base call cost”)

The cause of this is not a technical limitation of "this is not possible" but rather "Go does this poorly due to tradeoffs* it insists on".

C# trivially pins only select buffers or other data that need to be observed by whatever it is being called across FFI while GC is free to move and handle everything else (or, should the data need to be marshalled, it can be stack-allocated and pointer to such is passed instead, or it can be malloc and freed just like you can do in C). Years of work in this area made the impact of FFI as minimal as possible, with extra levers available to the user to completely erase the overhead (suppressing GC frame transition, using methods to allocate objects on the dedicated pinned heap, static linking for aot binaries, etc.).

Go's GC implementation is comparatively simple and has other inefficiencies such as very expensive write barriers.

*sometimes the tradeoff is just a product, which Go is, that was cheaper to produce, despite common beliefs

Yup, and far too much spinning going on in the runtime too.

(C# can handle millions of active tasks, and so can Elixir, though with greater overhead as it offers a different set of tradeoffs)

If only there was a high-level GC-based language with fast FFI and good performance, surely the industry would pick it based on the merits and not simply what is hyped...

Sadly its owners have tainted a couple of generations to avoid anything from them, and the way they manage their OS isn't helping to change the point of view.

That is how we end up using less capable alternatives.

It's funny because that's how I feel about Rust on this website.

Reply on news.ycombinator.com