Goroutines are not significantly lighter than threads
matklad.github.io
matklad.github.io
(Of course, in Go, the scheduler also weaves in the GC IIRC, so an apples-apples comparison may be difficult. Micro benchmarks are just not that useful.)
P.S.: this article seems to work under the assumption that 10,000 Goroutines is a reasonable upper limit, or at least it feels as though it implies that. However, you can definitely run apps with 100,000 or even 1,000,000.
Kernel context switches are pretty light compared to the following slowdowns due to cache.
I don't know which category epoll fits into, but it seems like it should be treated optimistically, since epoll is used for non-blocking I/O.
https://utcc.utoronto.ca/~cks/space/blog/programming/GoSched...
In epoll case, using RawConn.Read would look like this:
err := rawConn.Read(func(fd uintptr) bool {
nevents, err = syscall.EpollWait(int(fd), events[:], 0)
if nevents == 0 {
return false // try again
}
return true
})
https://www.usenix.org/system/files/login/articles/chen12-06...
Go provides go routines as a building block, it's up to you to use that or use something else ( reactor pattern, epoll ect ... )
Two points: a) native threads switching overhead is extremely low, probably lower than usermode Go thread overhead; b) the utility of usermode threads isn't to lower overhead.
It's a given that usermode threads introduce lots of overhead, if only because you're reimplementing the scheduling machinery that already exists in the kernel and that you're already paying the performance penalty for.
But theoretically, you could implement specially tailored scheduling algorithms specific for your app and gain an advantage there - kernel schedulers are general-purpose and can introduce latencies, starvation and other undesired special effects.
P.S. The Linux kernel also has interfaces for changing the thread scheduler algorithms. This gets you 99% of the way there - native threads + round-robin realtime thread scheduler gets you the performance benefits you want without having to rewrite schedulers and context switchers in usermode for every language runtime.
This is most likely very wrong.
In most apps, context switches would happen only due to kernel->user transitions anyway (such as an I/O syscall finishing or a timer triggering), so in most apps kernel context switches are indeed almost free.
To be significantly more efficient than the kernel there should be much more context switches than syscalls, which for non-synthetic programs is mostly only possible under heavy loads that can batch, with suitable I/O interfaces such as io_uring.
They don't even attempt to distinguish the size of the stack from the rest of the runtime.
for i := 0; i < 10; i++ {
go func() {
f(i) // sees whatever value i has when f() is called, usually 10
}()
} for i := 0; i < 10; i++ {
go func(i int) {
f(i)
}(i)
} for i := 0; i < 10 i++ {
i := i // new variable i for every iteration
go func() { f(i) }()
} for i := 0; i < 10; i++ {
go func(i int) {
f(i) // sees whatever value i has when f() is called, usually 10
}(i)
}Same situation in Python for example: https://stackoverflow.com/questions/54288926/python-loops-an...
Surely only if you store it indirectly, otherwise it'll be copied into the array.
Without measuring 10k, 20k, 30k of each and seeing how the memory usage changes, we are seriously comparing apples to oranges here.
Go's allocator will hold onto a decent chunk of memory just to amortize the cost of reaching out to the operating system. Just asking the operating system how much memory the Go application is using doesn't tell the whole story in this benchmark.
Is it 3x or 30x? The author certainly doesn't know based on taking a single data point from each application.
Goroutines are more memory efficient than full OS threads, but they're also cheaper to spawn and the Go runtime can optimize for certain common use cases very effectively.
I've also personally had serious difficulty in the past with getting Linux distros to let me launch tens of thousands of OS threads. Long before I run out of memory, I hit various limits that block the program from spawning additional threads. Some of them can be adjusted, but I never managed to spawn threads arbitrarily up to the amount of memory that was available... probably just my own failing, but that alone is reason enough not to spawn an unbounded number of OS threads.
I believe the article fails to account for, among other things, the virtual memory overhead that comes with sparse allocations for large maximum stack sizes, that would about double the numbers. With 2MB or so stack spacing, if you only use a single page of RAM per thread, you still need another whole page of RAM in the "page table" (actually a trie) and on linux those are not counted in the process RES memory usage.
That being said, creating thousands of native threads has lots of other pitfalls, including stack ones - would not recommend as a general strategy.
If haven't tried creating actual threads, but for test mapping stack-like staggered memory, I've had success with:
sysctl vm.overcommit_memory=1
sysctl vm.max_map_count=10000000
swapoff -a
> A thread is only 3 times as large as a goroutine. Absolute numbers are also significant: 10k threads require only 100 megabytes of overhead. If the application does 10k concurrent things, 100mb might be negligible.
If it's still not clear: if your application has a good reason to run 10k parallel tasks it's most likely doing something complex that requires plenty of RAM.
It's very unlikely that you really have to save those 100MB of RAM and at the same time you cannot rethink the architecture to stop using this level of parallelism.
And even so, that would justify using a coroutine library, not a whole programming language.
"RAM" is inaccurate in that sentence.
It's important to realize that the common knowledge "you can't have 10k threads" comes from 2 decades ago when we were running 32-bit CPUs. A thread may only need a couple pages of RAM; but it needs a whopping 1 MB (or more) of virtual address space (at least with default stack sizes). You simply can't start 10k threads in a 32-bit process, as you'd need 10 GB of address space. OS-level threads use a lot of address space and thus were unsuitable for the C10k problem back in the day. But in 64-bit world, 10 GB is nothing, the address space usage can almost be ignored and the RAM usage is much less bad. (but: RAM usage of page table entries also needs to be considered. the article forgot about this)
Of course, just because it works now doesn't mean that 10k threads are a good idea.
What happens when these threads or goroutines start doing real work (CPU, IO) concurrently ?
A few years back, when I was writing something for a presentation, I spun up 50 million coroutines, and arranged them in a ring with channels and passed a value all the way from each co-routine to the other, on Windows Laptop with 8GB of RAM.
I don't recall the exact amount of memory used, but my laptop handled it really well.
Now, the official implementation allocates a 2k-byte stack for each new goroutine. The initial stack could be much smaller in theory.
Typical programs will transparently grow stack on demand, both with native threads (memory not mapped until used) and goroutines / green threads.
Unlike green threads of typical runtimes, I don't know of native threading that releases no longer used stack space, until .. I guess the thread is destroyed, so sporadic large stack use on can blow up your memory.
They don't. Although if you know that may be a concern for your application you can easily enough `madvise` after calls which may map significant amounts of stack, in order to release it.
The article says:
> The workload is representative for measuring absolute memory overhead, but is not representative for time overhead.
The workload is definitely not representative of real memory overhead. What's worse, it almost can be, but it is very dependent on the pattern of stack usage and it may not be obvious that you are holding on to large amounts of stack.
Default stack size on linux is 8MB. If you don't use much stack, 10k threads might only need 200M of RAM. Or your threads might touch 2M of stack early in their lifetime, parsing recursive data structures or whatever, and you need 20G of RAM, because you're stacks don't get freed.
Realistically, when using green threading, I wouldn't give much thought before using 10K waiting threads.
With native threading, I would look for other reasonable solutions, because it's not worth the headaches, but in case they do turn out to be the most reasonable solution, I would be sure to appropriately lower the stack size to avoid OOM, budget at least a week for solving gotchas, and forever need to be vigilant about stack usage.
I've always felt that the biggest reasons that threadless concurrency primitives were adopted was to avoid the non-negligible cost of spawning threads and context switching between them.
But Rust's mpsc package is a good public example of the concept off the top of my head running on native threads. They're really not that complicated and a c++ version would be on the order of a few hundred lines.
I had been using the concept since the 90s, as RTOSes really love it. Pretty much every N64 game uses native threads communicating via "software FIFOs".
It's split into a read half and a write half so you don't have to worry about copy or clone, as an execution context will only have one or the other. I guess you clone the write half to give it to multiple producers, but that's not a huge deal.
The example here for instance doesn't look any more complicated than using go channels, and it's a full executable example you can start by clicking the run button.
tokio and async/await have little to do with channels (other than having their own async-aware versions).
If you need a real mpmc channel, crossbeam[0] provides one. The interface is similar to the stdlib's, but you can clone the receiver as well as the sender.
[0] https://docs.rs/crossbeam/0.8.0/crossbeam/channel/index.html
So the Go language is really giving you 2 things together that C++ doesn't have that makes concurrent programming easier. But arguably they could have just given you threads and channels and GC, and not goroutines. That is, leaving out M:N threading. (I thought gccgo did that a long time ago or still does?)
You can use threads and channels (which are just thread-safe queues) in C++, and follow a convention where you NULL out pointers after passing them across a channel. So basically you have single ownership.
That is all a manual process, which is OK sometimes, but when you add in threads and the possibility of nondeterministic bugs, then maybe it's easier to see why that style of programming is not super popular. Though I have done it for small programs and it works well.
If you want to be really minimal you could just use pipe() with each end in a different thread but the same process, even passing pointers over it.
Probably the bigger deal is the consistent networking libraries, and the consistent mechanisms for timeouts and cancellation that Go offers. I think that's difficult in most C++ programs using threads. (I'd be interested to know what C++ programs do that well)
On a project several years back, that's what pushed me toward async -- doing timeouts in a principled way.
Though to be fair to Go, I would say that memory corruption bugs can be highly non-local (from symptom to code fix), and race conditions less so. Bugs that have both properties are really painful and I think Go mostly helps you avoid the former.
If you do goroutines, you need to redo all IO yourself anyway, and that’s a good opportunity to implement things like universal support for cancellation and timeouts.
In short it gets to a point where it seems like you're only asking questions to prove someone else incorrect or even inferior - versus to actually further a conversation by contributing concrete evidence. This may not have been your intention, or I could be misinterpreting it, but in your shoes I would want someone to tell me they had this impression so I could consider it going forward.
I was under the impression this was context switching rather than memory usage, which I haven't seen much about when these models are compared.
Now that Go defaults to multiple threads it has lost all of that advantage. Goroutine switching and channel send/receive has to apply all of the locking that any multithreaded program has to use.
[edit] I should mention that there are other potential overheads if you use things like thread local variables as these may require more work from the GC to be collected. We are working on a new mechanism which will should be better in this regard and be a better API for many of he uses of thread locals.
When you think about memory in quantities of 1,10,100,1000 million units, a few hundred MB makes a big difference (e.g. Android devices).
And on severs - people implemented webservers like apache 25 years ago without coroutines and it ran with with very limited RAM.
This includes a 233 words (1864 bytes) private heap.
This likely wouldn't work for TFA (as it uses "active" threads) but it should go lower if you immediately hibernate the processes:
> The heap is then shrunken to the exact same size as the live data that it holds (even if that size is less than the minimum heap size for the process).
While I've not tested it, I assume a just-started process has nothing on the heap, so it'd go down to just the "overhead" 105 words (840 bytes).
Though that's at the cost of immediately allocating a new min-sized heap when the process wakes up.
But often times, they co-exist together to solve the problem. That's the case in Go. goroutines run on threads after all.
Op's could have used a better example like a web/tcp server handling 100k connections and showed the difference between coroutines alone, threads alone, and both combined.
Most HLL runtimes (that you’d ever want to use for writing a server) have some async IO completion mechanism using scheduler-reserved “async IO waiter” threads, exactly akin to netpoll. (Or they plug into the OS in a less-portable way that avoids blocking IO syscalls to begin with, like Node’s libuv.)
Or, if the language’s runtime doesn’t do it for you, then the popular connection-pooling abstractions that the language’s server libraries are built on top of do it for you, by spawning their own AIO completion threads. Jetty’s worker pool in Java, Tokio in Rust, etc.
Honestly, who’s out there writing code in 2021 where there are worker POSIX threads, and those threads are (directly or indirectly) calling read(2)?
I don’t write web services, so I have low O(1) things to communicate with, for which threads with blocking read work OK. They are not perfect, as there is no cancellation, but working around lack of cancellation in a couple of places is less costly than bringing dependency on a relatively fresh Rust async ecosystem.
.NET Orleans, for example: technically cooperative-scheduled tasks/fibers, but where yield points (i.e. generated state-machine transition points) are decided at compile-time; and where the inability to place such a yield-point within a soft-real-time-bounded distance from the previous one is a compile-time error. You don’t have to place the yield points yourself (like explicit async code), but they’re right there to see when you debug or disassemble the program, and you can use pragmas to adjust where or if they’re placed.
Technically the Erlang runtime is this way as well (cooperatively-scheduled, with “explicit” yield points) but since one of those explicit yield-points is use of the CALL or RET ops—and since there are no looping ops, only tail-CALLs—it’s kind of hard to write an explicit atomic-CPU-blocking hard-realtime task-segment in Erlang. (You can, but you have to unroll all your loops. Which Erlang has no tooling support for. Much easier to just push the problem to C through a NIF. At least the Erlang scheduler has explicit design support for these long-running blocking NIF operations through “dirty schedulers”, so it’s not like you’re breaking the runtime’s operational tolerances by trying to do hard-realtime things in it.)
The post is very direct about looking only at the memory usage, and literally says that using the results to reason about overall performance would be wrong.
I see how the article could be read as “goroutines bad, threads good”, as it doesn’t go to extraordinary lengths to prevent that. But I prefer to cater to a careful reader, and not add loud disclaimers repeatedly.