Hacker News Reader

The implementation of the coroutines might be complex, but their use is very straightforward -- the code looks almost exactly the same as the blocking code would, just with the awaits in there.

As for the number of threads, when I use Boost ASIO with coroutines I often end up with multiple threads servicing a pool of coroutines, so if there is shared state between them then there is still synchronisation. I use channels implemented on top of eventfd to help with that.

When I converted some network code from callbacks to coroutines the code ended up about 1/3 as long and was far simpler and easier to understand. It also fixed several bugs I simply couldn't find.

The reality is that the callbacks are far more complex to use than the coroutines.

Yup, we used to let our designers loose on coroutines as a way to script Game AI. If designers can use them safely then I'm sure the average programmer can too :).

I found them useful when using the Boost Asio C++ networking library.

I needed to read some bytes, make a decision, read some more bytes, make another decision, etc. This started out as callback hell since every read was asynchronous, which was very painful, but after using the coroutine support I was able to write the code as if it was synchronous and the logic became a lot easier to follow.

And the coroutines weren't that hard to use, even lacking proper language support. I could basically write something like async_read(socket, buffer, yield_context);. The coroutine yielded behind the scenes and the buffer was populated once control flow was returned to the next line (from who knows where!)

> 2) Have a single-threaded program, where that single thread runs an event loop whose job is to react to external events by invoking a callback function that has been registered. While it doesn't require the same kind of complex synchronization that preemptive threads do, the inverted control structure of this approach requires your own control flow to thread awkwardly through the system's event loop, leading to a maze of event callbacks.

Co-routines and callback based programming are great for dealing with I/O and for offloading "work" to black boxes, those black boxes can very well be other threads, CPU cores, hardware, or another computer.

When the work is complete, your callback is called, or the co-routine continues.

It is perfectly possible to be juggling multiple tasks this way, but have all the results appear in sequential order on one thread, and not have to deal with the complexity of real multi-threading.

This only works for tasks that are easily offloadable, you need VERY clean data boundaries, minimal shared state, and well defined points where you hand off control.

In other words, a well engineered program. :)

Concurrency is about how you organize a solution really more than anything. Whether those concurrent tasks execute in parallel or not is a different consideration. Many authors and experts conflate these terms.

You can write concurrent code with pthreads. It may happen to execute in parallel but it doesn't have to. In fact, I believe the early SunOS/Solaris pthreads had a 'green' scheduler where it could all be done in userspace (this is from my own memory nearly 20 years ago). This made them basically "stack switchers" which is what some lightweight coroutine libraries end up doing.

Having written heavily callback-driven code. Threading contexts manually, thinking about object lifetimes, I can say that coroutines have a much cleaner flow to them and can make it easier to write and later read and debug programs than crazy callbacks and contexts.

Object lifetimes are one thing coroutines help with in a big way. I think it's why Rust is so attractive to so many.

Look at plan 9's libthread for example: https://github.com/9fans/plan9port/tree/master/src/libthread

It allows one to take advantage of nonblocking sockets as a single "kernel execution entity" mechanism across many userspace threads.

This approach works pretty well. The Mongrel2 webserver clearly borrows this approach in their task library:

https://github.com/9fans/plan9port/blob/master/src/libthread...

https://github.com/mongrel2/mongrel2/blob/master/src/task/38...

> Having written heavily callback-driven code. Threading contexts manually, thinking about object lifetimes, I can say that coroutines have a much cleaner flow to them and can make it easier to write and later read and debug programs than crazy callbacks and contexts.

This is the biggest reason I vastly prefer using coroutines in my Javascript projects (by way of `redux-saga` if there's heavy lifting, or just async/await otherwise).

> Concurrency is about how you organize a solution really more than anything.

You hit the nail on the head here. IMHO, business logic should be expressed procedurally, whereas driver code in an event-loop driven environment (like V8) is more suited to a functional style. Coroutines let you do both, quite cleanly.

Thanks for the links, btw, I've been wanting to dig through other code like this in more established languages. :)

The fundamental difference between single threading, coroutines, and multi threading:

Single Thread: 1 thread of execution, 1 execution stack

Coroutine: 1 thread of execution, multiple execution stacks

Multi threading: multiple threads of execution, multiple execution stacks

FYI each coroutine is normally considered its own thread of execution (which is a distinct thing from a kernel scheduled thread).

Coroutines are distinct from threads as they normally lack a scheduler (the decision to which coroutine run next is either made statically ot explicitly by the coroutine); coroutines meant to be used on top of an userspace scheduler are often called fibers (or simply userspace, or green, threads).

You can have multiple threads with coroutines. However you generally only have one thread per CPU/hardware thread, and use async io within that thread.

This is the general architecture of nginx for example (though it doesn't use coroutines, just callback based async io).

Event loop + callbacks allows you to fully utilize any given thread (meaning that while there's outstanding I/O or other long-running operation, the thread isn't idling waiting for it to complete, but instead performing some useful work). This is completely orthogonal to multithreading - if you have multiple threads, there's still benefit in fully utilizing them, and so each thread gets event loop and callbacks. It's true that some popular frameworks in this vein (notably, Node.js) combines event loop with single thread per process, but it's not a prerequisite. In C#, async/await is commonly used with multiple threads, for example in ASP.NET web apps.

Or 3) you do like golang and erlang and build really lightweight green threads so you get the programming model of approach 1 and nearly the same speed as number 2.

This of course requires extra effort to prevent blocking of OS threads and in the case where the OS expects thread locality. But IMHO it's the superior approach.

The best solution would be if OSes themselves would provide really lightweight threads. And I suspect that they will, eventually.

I haven't done Erlang, but in my experience Golang has much the same problems as 1) so I'm not sure I'd agree. +1 to lightweight threads, tho.

Phoenix/Elixir can handle 2 million websocket connections on a single box. I would agree myself :)

http://www.phoenixframework.org/blog/the-road-to-2-million-w...

Wouldn't you need something like 40 NICs to do that?

I believe you are referring to running out of ports, as 65k available ports * 40 ~= 2M.

The limitation is actually on a quad tuple: {src_ip, src_port, dest_ip, dest_port}

As such, the actual limit is outrageously high and isn't really a problem. Interestingly enough, the referenced Phoenix benchmark overlooked the dest component and hit a limit as they had only 40 or so machines to use in the test. Had they used multiple ips on the server it would have given them a lot more breathing room.

EDIT: Multiple IPs != Multiple NICs. You can assign multiple IPs to a single physical interface. Also, they could have used multiple ports to accomplish the same effect.

Good to know. I thought you were always limited by the outgoing port range to 65k - (used ports) good to know.

So they were limited by the testing cluster or the tested server?

They were limited by the testing cluster, but it was pointed out to them after the fact that they could have used multiple IPs or multiple ports on the tested server to go even higher.

The problem with this approach is that as soon as you need to call out to a library written in a different language, it has no understanding of your lightweight green threads - precisely because it is currently a language/VM abstraction, not an OS abstraction, so everyone rolls their own. Furthermore, existing languages (notably, C) produce a legacy burden that'll remain even once OSes do start to add such threads.

For what it's worth, Win32 has offered a kind of green thread (fibers) for a very long time. Adoption has been minimal, even for other Microsoft products (e.g. .NET doesn't really support them properly).

2) is concurrency, 1) is parallelism; there are various programming models for multiple threads. The basic approach of mutual exclusion (=shared memory, locks, barriers) is just one of them, another approach would be message passing / actors (queues, ZeroMQ etc.). You can mix the two, and the latter typically is implemented on top of the former, because hardware does not have message passing facilities. Some other approaches don't really fit that scheme (e.g. TBB).

Choosing an approach in this field is not always obvious, though sometimes it is. E.g. many stupid server workloads are bound on I/O concurrency, so green threads are the best fit.

However, as soon as CPU scaling is required, multiple independently schedulable kernel tasks are required -- you either have to fork/CreateProcess or spawn threads. This does not necessarily imply that larger parts of the application have to be aware of this. E.g. it's entirely possible to plug a behind-the-scenes CPU thread into an event loop.

I'll take a crack at this - coroutines are not about performance. They are for converting callbacks to a single thread of execution that's easier to reason about (which could themselves be embedded in other threads).

Imperative logic is equivalent to a state machine but again much easier for humans to reason about, that's why developers got so much more done in the past with synchronous execution and macros than they do today as we struggle to build even the simplest user interfaces. I'm hopeful that we can get all of the benefits of 1980s cooperative threading but that the decades of experience since allow us to avoid the pitfalls.

You can also have mixtures of those approaches in languages which support multiple threads. E.g. you could have one thread which runs an eventloop and async operations and possibly coroutines. And other threads where a pure blocking programming model is used. From these threads you could e.g. post some work to the eventloop thread and wait for it to complete.

The Future/Task abstractions that are found in multithreaded programming languages (like C# or Java) also reflect this possibilities: E.g. they allow to specify on which thread a continuation should be run instead of just assuming the current thread. They also need synchronisation internally, because multiple threads might want to use the Future object.

From my personal experience the things that work well are pure singlethreaded models with an async style (like in Javascript/node.js) or multithreaded but pure synchronous models without callbacks (as popular in Go). The mixture (multithreading with callbacks, which might be invoked on any thread) often leads to confusion how an API should be used correctly.

Coroutines may also require mutexes. Usually they are very coarse-grained, but the usecases for coroutine mutexes and wait queues exist. These are not pthread mutexes, of course, they would be part of the coroutine runtime or hand coded.

For example, you could have coroutines that write to a shared resource (such as a socket) where interleaving the data from two sources would be a problem.

You can have 2) with the additional complexity / upside of multiple threads