The Rust standard library no longer has any scheduling baked into it

The less the language actually specifies, the more likely you are to encounter the situation where two Rust libraries can't work together because they actually work under Rust-Alpha (some twiddling of the knobs) and Rust-Beta (some other incompatible twiddling of the knobs).

I don't know that this is a problem for Rust yet, but it is the first announcement from the Rust team which has actually dropped my level of interest in the language. Is there recognition that this is a tradeoff and not something you can think of as purely a "feature"? Do they realize that this was a huge step towards potential fragmentation of the language? (Not just in this particular decision, either, but the general mindset that leads towards this decision leads to fragmentation.)

With Rust I'm sure they'll drop M:N, if not now then eventually, like essentially everybody else in CS history. It seems great on a superficial level, but when you really start caring about performance and latency and fairness it's the pits.

My question is when Go will drop it. Imagine how easy it would be to have C call Go code if Go didn't have its own threading and segmented stacks; simply push the arguments and call/jmp. Maybe reference a few objects. None of that allocating a stack, locking a goroutine to a thread, stealing work to another thread, etc. You could even embed Go into a C program instead of having to do it the other way around, and you wouldn't have the problem of adapting existing programs or rewriting them in Go.

But I bet Go will continue with M:N and segmented stacks for a decade because it seems like they actually want it to be difficult to interface with anything else. Like Pure Java, they want you to rewrite everything in Go instead of just using some existing library.

(Also, yeah, Rust is pre-release and is definitely not suitable for people who don't like making minor changes regularly if they're tracking master (or less minor changes occasionally if tracking the 0.x releases).)

That said, breaking changes are becoming less and less common. The language syntax has been mostly stabilized and most changes are going into corner cases and managing modularity, and much of the possible experimentation has been fenced to the post-1.0 release.

But this allows Rust to do bleeding edge research instead of finalizing on a strict specification before knowing if the design will work in practice. It is also very refreshing to see programmers admit that their choice was wrong and not insisting on staying with a bad choice that was made earlier.

Consider Rust to be a practical research project, at least for now.

If operating systems supported something like scheduler activations (a 20 year old technique), then this becomes less of a problem. The gist of it is that every time the kernel thread would block, or gets scheduled, instead of returning back to where it was executing, it upcalls into a user level scheduler which can then choose to schedule user threads. Its a shame that this technique isn't more common.

* IIRC each new process consumes 284 bytes so it's not "spending lots of memory on creating stacks for your green threads"

* Synchronization is a non-issue as everything is message-passing.

* Disk I/O is blocking, but the VM has dedicated disk threads.

* Dealing with 3rd party code is a problem, which is why relatively few libraries exist for Erlang. However, there is some hope now that NIFs (native interface functions, i.e. C code) can interact with the scheduler, i.e. report how much time has been used and return control if necessary.

You are right that "Building a correct highly concurrent scheduler is no easy task." The Erlang scheduler was not an easy task, and has received a ton of work from extremely talented engineers over the course of decades. Definitely worth a look to see how N:M can be made to work well.

1. For the scheduler, we use the JDK's superb and battle-tested ForkJoinPool (developed by Doug Lea), which is an excellent work-stealing scheduler, and continues to improve with every release.

2. For synchronization, we've adapted java.util.concurrent's constructs (we use the same interfaces so no change to user code) to respect fibers, but users are expected to mostly use Go-like channels or Erlang like actors that are both included.

3. As for disk IO, Java does provide an asynchronous interface on all platforms, so integrating that wasn't a problem.

4. Integrating with existing libraries is easy if they provide an asynchronous (callback based) API, which is easily turned into fiber-blocking calls. If not, then ForkJoinPool does handle non-frequent blocking of OS thread gracefully.

All in all, the experience has been very pleasant: callbacks are gone and performance/scalability is great. Things will get even better if Linux will adopt Google's proposal for user-scheduled OS threads, so that all code will be completely oblivious to whether the threads are scheduled by the kernel or in user space.

Regarding performance, Linux does have a very good scheduler (unlike, say, OS X), but while there's little latency involved if the kernel directly wakes up a blocked thread (say, after a sleep or as a response to an IO interrupt), it still adds very significant latency when one thread wakes up another. This is very common in code that uses message passing (CSP/actors), and we've been able to reduce scheduling overhead by at least an order of magnitude over OS threads.

I would summarize this as follows: if your code only blocks on IO, or blocks infrequently on synchronization, then OS threads are quite good; but if you structure your program with CSP/actors then user-space threads are only sensible way to go for the time being.

[1]: https://github.com/puniverse/quasar

> A lot of 3rd party code doesn't work great with userspace threads.

Agreed, especially since third-party code often calls standard library functions (`connect()`, `read()`, etc.) instead of the magic ones that are aware of your concurrency model. At OkCupid, we have our own database driver and RPC implementations that know how to use events for networking, and it sucks to have them in our codebase.

> The final nail in the coffin for me was disk IO.

Why not make it "somebody else's problem"? My thing uses libuv, which provides a common API and uses a thread pool or new async APIs under the hood. At OkC most filesystem access transparently goes through a separate process — it's just network access from the app's point of view.

I guess what I'm trying to say is that if you can solve the problem once, you can make a nice abstraction on top of it and never think about it again (unlike dealing w/ third-party code and scheduling).

Either way, I'd love your input on what I'm doing so far, you have way more experience.

I went through the callback-hell problem myself and came to the conclusion that swapcontext is the only way to go. I am starting to begin a large personal project that will be entirely based on swapcontext model (but using boost::coroutine), so I would like to know more from your experience before I invest into possibly the wrong model. Since you haven't explained your solution in-detail, I have a question:

Did you use multiple N:M schedulers (each owning an isolated group of os-threads) or your entire process uses a single N:M threading library that owns all os-threads of the process?

Here is what I am doing:

My project does lot of disk and network IO. So, IMO, being able to keep CPU, Disk and Network busy is my design goal. Having a single N:M scheduler that handles all threads makes it very difficult to understand/analyze/manage the system performance, so I made the choice of having multiple N:M schedulers (that communicate through message passing), each deals with separate resources of the system. For example, os-threads of a N:M cpu-scheduler will only do compute work and when a disk (or network) io is necessary, the greenlet/task/whatever will be queued into the disk-io-scheduler (or network-io-scheduler). When the os-threads of disk-io-scheduler (or network-io-scheduler) finish the io, they queue the greenlet/task back into the cpu-scheduler.

This IMO keeps managing/analyzing/tuning the system performance and complexity little easy. For example,

> All of a sudden you're forced to write a userspace scheduler and guess what it's really hard to write a scheduler that's going to do a better job that Linux's schedules that has man years of efforts put into it. Now you want your schedule to man N green threads to M physical threads so you have to worry about synchronization. Synchronization brings performance problems so you start now you're down a new lockless rabbit hole. Building a correct highly concurrent scheduler is no easy task.

Agreed, writing a good scheduler that deals with mixed workloads is very challenging. In my model, I am hoping that a poor scheduler wouldn't be an issue because, os-threads in the cpu-scheduler are only competing for cpu-bound tasks; similarly os-threads of io-scheduler are only competing for io-bound tasks. The real scheduling between io and cpu operations is still left to the kernel (because kernel decides to pick which scheduler's os-thread to run.) From kernel point of view there are threads that only do IO work or threads that only do CPU work. I am not yet sure if kernel schedulers are optimized for such workload or not.

> A lot of 3rd party code doesn't work great with userspace threads. You end up with very subtle bugs in you code that are hard to track down. In many cases this is due to assumptions about TLS (but this isn't the only reasons). In order to make it work you now can't have work stealing between your native threads and then you end up with performance problems and starvation problems.

My design-solution to this is to segregate 3rd party code into a fixed set of threads that do not ever participate in N:M threading library. Would that solve the above issues?

> The final nail in the coffin for me was disk IO. The fact is that non network IO is generally blocking and no OS has great non-blocking disk IO interfaces (windows is best but it's still not great). First, it's pretty low level, eg. difficult to use. You have to do IO on block boundaries. Second, it bypasses the page cache (at least on Linux) which in most cases kills performance right there. And in many cases this non-blocking interface will end up blocking (even on windows) if the filesystem needs to do certain things under the covers (like extend the file or load metadata). Also, the way these operations are implement require a lot lot of syscalls thus context switches which further negate any perceived performance benefits. The bottom line is that regular blocking IO (better yet mmaped IO) outperforms what most people are capable of achieving using the non-blocking disk IO facilities.

Yes, lot of disk/file-system operations do not have non-blocking equivalents. This is one of the main reasons for having a separate disk-io-scheduler (with its own thread-pool) in my design. In my model any blocking-only operation is made asynchronous by queuing the operation into a dedicated io thread pool and let it inform me when the operation is completed. This gives me flexibility in tuning the IO parallelism specific to my needs. For example, SSD can handle more concurrent operations than a HDD, so I can create an ssd-io-scheduler with more number of threads than a hdd-io-scheduler.

It would be great if you could share your thoughts on my analysis. Thanks.