Comment by the_duke | Hacker News Reader

Comment by the_duke | Hacker News Reader

Parent Full thread

I'm personally not a big fan of Go due to the weak type system, with lack of generics as the main offender. And as a result, the ubiquitous `interface{}` pattern, stringly typed annotations, ...

BUT Go offers simplicity, incredibly easy concurrency, decent performance, a good standard library for networking/http services, very quick compile times and a stricter type system than many other languages like Python/PHP.

For many teams, especially coming from more dynamic languages, Go fills a sweet spot of performance, concurrency, type safety and ease of use.

I love Rust, but it is much more complex, boilerplate heavy, adds more friction, is harder to learn, and concurrency is much harder to get right, even with the upcoming async/await feature. Not to mention the immature ecosystem, that will still need years of catching up to achieve parity with Go.

Kotlin is also a great language, but has it's own downsides. You need to learn and understand Java (due to the many Java dependencies that you will pull in), there often will be a mismatch between Java libraries and idiomatic Kotlin code, concurrency is harder than in Go, and the JVM is a much more complex beast then the minimally tunable Go runtime.

Go definitely has a place, even though it often wouldn't be my own first choice.

> the ubiquitous `interface{}` pattern

I’m totally biased because I’ve worked on and with go for a decade, including reviewing all sorts of Go we write at Google, but I just don’t get this criticism. It’s just not something I see in the code bases I work on. I admit that I may be working on different kinds of programs to what some others do, and I don’t want to deny anyone their legitimate experiences, but my personal experience has been that this is rarely a thing, if ever.

It can be found in many libraries in the ecosystem, and is common enough in the standard library.

The lack of generics just severely restricts the expressiveness of the type system.

I think the best way to work around this is use of code generation, which is a great tool for internal, controlled environments, but very cumbersome for open source libraries.

I don’t see much, if any, generated Go code at Google. Not to say that it doesn’t happen but it’s not on my radar.

Here is an example: https://github.com/google/gvisor - https://github.com/google/gvisor/tree/master/tools/go_generi...

It was useful enough that someone made a standalone package: https://github.com/mmatczuk/go_generics

Kubernetes makes heavy use of generated code, as workaround for lack of generics.

They even gave a FOSDEM talk about this, with a very interesting title.

https://fosdem.org/2019/schedule/event/kubernetesclusterfuck...

Aren't http or grpc apis autogenerated? You'd be crazy not to use a generator for those.

sync.Map?

I don’t have the experience you do, but I feel like it comes up a lot when working with dictionaries and json.

I've worked on a million line Go code base and a dozen open source projects and many typical backend services, and interface{} is actually quite rare outside of something like json serialization or string printing - i.e. fmt.Printf(msg string, args ...interface{})

I almost never see code that application developers write using it, and usually if someone is writing that code, it's wrong, and I help them figure out how to avoid it.

Usually you can avoid it by using closures or interfaces.

Just depends on the person. Some people like to copy and paste 50 times, some people use interface{}.

Functions are a thing. Interfaces are a thing. I have literally never, in 6 years of full time go development, just copied and pasted code for the purpose of making it work with a different type. Nor has anyone I have worked with in that time (to my knowledge). I have seen two or three libraries written that use interface{}, but these are by far the exception.

Just depends on your application domain. I never copy/pasted code and could count the number of times I wrote `interface{}` on a single hand.

> the ubiquitous `interface{}` pattern

Ubiquitous where? It's the equivalent of `Any` in Scala (and other similar strongly typed languages with unified type hierarchy) and should be strongly discouraged in production code.

Standard library APIs like context.Context's WithValue/Value, sync's Map/Value/Pool, stuff like the sort or container/heap packages. Some of them are obvious "really wanted generics, but wasn't important enough to get the special built-in treatment" cases, some aren't.

Most logging libraries I've seen follow the pattern from fmt.Println etc to accept arbitrary `interface{}` values to be logged instead of actually using some Stringer or Formatter interface in their actual method signatures.

Context's WithValue will be an interface even if go supported generics. It stores anything, not values of a specified type. Same with fmt

Yeah, that's what I meant with "some aren't", I was mostly listing stuff to support the idea that `interface{}` itself is ubiquitous and not some super-niche edge case that never shows up in real code.

async/await is async io, this is something js is known for since it doesn't actually require parallelism.

Getting concurrency _with_ parallelism right is a lot easier in rust than it is in go. You need to write thread safe code in both of them, but you get zero support from your compiler in go.

Go provides channels for safe communication between goroutines (which are distributed across multiple threads).

You can use channels to pass mutable objects, which are then shared between threads. The passing itself may be safe, but the resulting situation isn't.

The Rust compiler won't let you do that.

You can mutate objects from multiple threads in Go without using channels.

The Rust compiler won't let you do that either!

I know. I just didn't see why you were bringing channels into it. It's not as if it's some loophole involving channels that's the issue here.

Rust prevents data races with the `Send` and `Sync` markers, but that's a long way from making concurrency and parallelism easy.

In Go, as long as you are not sharing data, you don't really have to care about whether something is concurrent or parallel, the runtime takes care of it for you. You just use goroutines, IO transparently happens on a dedicated threadpool, CPU heavy tasks are spread out over the available cores.

As long as you restrict sharing to sending owned data over channels, you are mostly in the clear. Of course sometimes performance concerns etc mandate more traditional tools (atomics, mutexes, etc) and there the compiler won't help you anymore, but I'd argue that is the exception in Go rather than the norm.

You can often solve problems with a actor-system inspired design. (learning Erlang/Elixir will probably make you a better Go developer!)

In Rust, you still have to care a lot about what you do. Things like tokio::fs can handle file system IO for you, but you need to ensure that code you are calling does no synchronous IO. Also, CPU heavy workloads should be offloaded to a dedicated CPU thread pool, since the tokio runtime is not intended for juggling CPU intensive tasks in addition to the async IO. This is fine for Rust, a language that wants to give you insight and control, but it is still much more demanding of the developer.

> You just use goroutines, IO transparently happens on a dedicated threadpool, CPU heavy tasks are spread out over the available cores.

In Rust you can do that too! Just use threads. That's what Go does: it just uses threads.

Async I/O is there if you need better performance than what Rust—or Go—can provide. If you don't need that performance, then don't use async I/O.

Go uses async IO under the hood (for network IO at least). The exposed API is blocking, but the underlying implementation uses async IO and transparently schedules Goroutines across different OS threads.

As you know, Rust doesn't currently have stable support for a similarly high-level interface to async IO. See e.g. the following reddit thread from 3 months ago:

https://www.reddit.com/r/rust/comments/ausuld/what_is_the_cu...

Using async I/O in userspace to implement the sync I/O interface of goroutines is equivalent to what the kernel does to implement sync I/O for normal threads.

Kind of, but with the difference that Goroutines are lighter than kernel threads, so you can run more of them.

Not as much as you'd think. You can run thousands of kernel threads on Linux.

https://golang.org/doc/faq#goroutines

"It is practical to create hundreds of thousands of goroutines in the same address space. If goroutines were just threads, system resources would run out at a much smaller number."

More generally, Go is hardly the first language to implement userspace threads backed by OS threads. Erlang famously does this, for example, and so does GHC: https://en.wikibooks.org/wiki/Haskell/Concurrency

In response to your other comment, the whole point of Goroutines is that you can use them pretty freely without (usually) having to worry about how many there are. Let's say you're writing a server, and that handling one of the requests requires completing 10 tasks, each of which can run in parallel. In Go you can just spawn a Goroutine for each of those tasks. But if you were using real OS threads, then that implementation could easily end up spawning more than a few thousand threads, when the server was under significant load.

That FAQ answer is misleading. Linux supports thousands of threads just fine. If you have hundreds of thousands, you probably want true async I/O instead of having stackful threads at all.

> But if you were using real OS threads, then that implementation could easily end up spawning more than a few thousand threads, when the server was under significant load.

There's no inherent problem with spawning a few thousand threads in Linux.

The FAQ is talking about “hundreds of thousands of threads”. It doesn’t make sense to respond by saying that Linux is fine with “a few thousand threads”. Similarly, I was talking about a scenario where a server would end up spawning more than a few thousand threads.

>If you have hundreds of thousands, you probably want true async I/O instead of having stackful threads at all.

This is a completely baseless assertion, and really makes no sense at all. "Async IO" in itself doesn’t even offer a means of exploiting multiple CPU cores. If "async IO" were all that was required, people would be sticking with Node.js for heavy loads.

>even 2kB of stack per thread is 195MB just from thread stacks.

You say this as if 195MB were a large amount of memory for a server to be using! Obviously, if the memory overhead of running 100,000 Goroutines is only 195MB, then memory limitations are not going to prevent you from running 100,000 Goroutines.

Linux is fine with a few hundred thousand threads too. I didn't mention it because it's significantly suboptimal to run that many OS threads or green threads like goroutines. If your scalability needs are that high, all the other little inefficiencies in Go, such as the lack of compiler optimizations and GC, are likely going to end up dominating more than threading models. We usually talk about high scalability as C10K; C100K is an extreme case.

> "Async IO" in itself doesn’t even offer a means of exploiting multiple CPU cores.

Sure it does. Multithreaded epoll is the standard solution for such things; that's how nginx is multithreaded, for example. Node isn't multithreaded, but that's a Node problem, not an async I/O problem.

> then memory limitations are not going to prevent you from running 100,000 Goroutines.

100,000 Linux kernel threads with 8kB kernel stack + 2kB user stack is only 1GB, which is likewise tractable.

And that 2kB assumes that the goroutine stacks remain small, which is not always the case. They can and will grow based on the dynamic behavior of the program.

You point out yourself that Linux kernel threads use more memory. For this and other reasons (e.g. slower context switches) you cannot spawn as many Linux kernel threads as you can spawn Goroutines.

The rest of your comment is just a repeated insistence that "Go doesn't scale" without a shred of evidence in support. You really seem to have some kind of beef with Go that's clouding your judgment in this area.

>Multithreaded epoll is the standard solution for such things

Multithreaded epoll is exactly what Go provides. Rather than having to manage thread pools yourself, the Go runtime does it for you by distributing logical threads of execution over OS threads.

>We usually talk about high scalability as C10K; C100K is an extreme case.

You seem to be assuming one Goroutine per connection. As I said before, a more plausible scenario for 100,000 goroutines is C10K with 10 goroutines per connection.

Sure, technically there is little difference, but my point was is is much easier to use for the developer. The complexity is hidden from you and the choices are largely already made by the runtime.

Sure, modern Linux handles 10k threads just fine, but where Go shines are middle-level network services or microservices where you can easily have 10+ additional tasks for each incoming request. Using real threads quickly becomes unfeasible, and also more cumbersome (passing around threadpools, which threadpool do I use for this task?, do I really want to do those two things in parallel and eat up an extra thread? the cancellation issue, ....).

Also in Rust you'd need to rewrite a lot of code when switching from sync to async, and still need to delegate things to a threadpool since not every dependency will be async enabled.

When the ecosystem stabilizes Rust will be a great choice for the lower stack libraries and servers, where the performance matters. But for many projects the increased complexity, for moderate performance gains, would be a hard sell.

Do many Go programs really have 100,000 goroutines simultaneously? I'm certain that's quite rare.

Well, even if you do not do any parallel work for each request in your service, each request in itself is a goroutine. So depending on the load of that service, you might reach such numbers, and more.

Working as part of Juju project, I can tell you for certain that the answer here is yes, very much so.

If you have over 100,000 threads, then you are at the point of scalability where you probably want more than what Go can provide. Async I/O would likely be better for your use case, because even 2kB of stack per thread is 195MB just from thread stacks.

People who think Rust is "complex" and "boilerplate-heavy" have never had to deal with C++, or heck, even Java. (Though I'd guess we would actually find plenty of C++/Java code within Google itself.) The only reason one could fairly give for that sort of statement is that Rust - the language itself, not just the ecosystem! - is still quite far from mature. But it's going to have a great future once some of the "tweaks" that are in the pipeline right now reach stability. Async is one of those of course, but there are more.

Edit: And to be clear, I'm not a Go-hater! In fact, Go comes out-of-the-box with a very nice, high-performance concurrent GC that makes it highly appropriate for some scenarios where Rust would not be a good fit at all.

I've written 100k+ loc in go and i've had to use interface{} once. I would like a coalesce operator though.

Reply on news.ycombinator.com