Hacker News Reader

Comment by overgard | Hacker News Reader

Probably not a popular opinion but: I'm not really a fan of all the user-space async stuff that's become popular (IE: the nodejs/tornado style of single threaded apps with non-blocking IO).

Remember Windows 3.1 or old Mac OS with it's cooperative multitasking where you had to explicitly yield or risk freezing up the entire computer? It's basically the same thing reinvented with a nicer brand. I know I'm dating myself here, but it's really the same model. We came up with preemptive multitasking for a reason: because it's more robust and efficient.

Threads are dangerous, but basically you can avoid the danger for the most part if you avoid sharing objects across threads, and if you do that then with blocking IO your code flows linearly and you don't have to deal with callback hell.

View on HN

> It's basically the same thing reinvented with a nicer brand.

Not at all. First, don't confuse preemptive scheduling with time-slice scheduling. All "true" user mode threads are preemptive (Erlang, Go, Quasar). Not all of them employ time-sliced preemption. In fact, when we implemented Quasar on the JVM we had time sharing but then took it out because it didn't gain us anything (other than increased implementation complexity). The reason is that fibers in interactive applications follow a certain pattern that always entails frequent blocking. You are right, though, that some types of computations do not fit well with the user-mode threading model -- long, CPU intensive computations. Those are best left for kernel threads that do time sharing well. Languages that don't give you access to kernel threads might implement time sharing for their lightweight threads in user mode (e.g. Erlang).

The parent was comparing the event model (e.g. callbacks everywhere) with a thread/process model; not comparing user-mode and kernel-mode threads. So I didn't follow your point.

Also, erlang doesn't use time slices, it counts "reductions" (essentially function calls, though I don't think it's always exactly one function call). It also penalizes processes more if they send to a mailbox that is already very large.

Erlang is considered truly pre-emptive, because (assuming you aren't writing your own functions in C or something) a function can't loop or use any operators or really do anything without potentially being pre-empted.

Go is generally considered partially pre-emptive, because it only pre-empts on a function call or during a memory allocation. You can write a simple loop that doesn't terminate, and it will never get pre-empted.

> Erlang is considered truly pre-emptive... Go is generally considered partially pre-emptive...

Well, in Quasar we started taking the "fully preemptive" route, but we saw that threads fall into two categories: those that block very often, and those that don't. Because the JVM, unlike Erlang and Go, gives you access to kernel threads, too, Quasar will simply warn you if you're using a fiber for a CPU-intensive operation that doesn't block often. Using reductions didn't work out so well because a "forcefully preempted" fiber still wants more CPU, which normally means it's doing something wrong.

Going partially pre-emptive is a reliability hit though. One errant process/thread/fiber can bring the system to a halt (or at least one hardware thread).

Erlang is designed for reliability, not CPU efficiency for CPU-intensive tasks.

Go actually does give you access to OS threads in the runtime package in the sense that you can easily reserve a thread for the current goroutine via http://golang.org/pkg/runtime/#LockOSThread and similarly can give it back up to the scheduler with runtime.UnlockOSThread()

You are wrong about Erlang. It's threads ("processes") are not preemptive and must explicitly yield for another thread to execute. Most of the time the VM handles it for you, every few hundred bytecode instructions it checks if another thread should run. But if you write a C extension, call sleep(10) then the Erlang VM will twiddle its thumbs for 10 seconds.

It's not a corner case -- it means any call to a C library needs to be carefully vetted so as to not cause an overly long computation or blocking because just a single one will totally ruin your concurrent experience. ALL VM:s using user mode threads/fibers suffer from that problem.

Real kernel threads doesn't have a lot of overhead and are almost always a much better choice.

> Most of the time the VM handles it for you

Yes, and that's the definition of preemptive. There's nothing explicit about it if the VM handles it for you. What you're saying is that it cannot preempt kernel-thread blocking code.

> Real kernel threads doesn't have a lot of overhead and are almost always a much better choice.

Well, as usual, that depends. A kernel thread task-switch is 20-50us, while a fiber task-switch is a couple nanos at most; that's a 1-2 orders of magnitude difference.

Sorry, I meant to say a couple hundred nanos.

That's just wrong. The definition of preemptive multitasking isn't that it works most of the time. A definition that lax would mean almost any cooperative threading system would be counted as "preemptive."

Also can you please find a reference that shows (Linux) kernel threads being 1-2 orders slower than user threads?

It's a little less dangerous when you only need to cooperate with yourself, but yes, it does mean you need to be very careful that you don't have any means for some requests to take an unexpectedly long amount of time.

In MacOS if you grabbed a window title bar for too long with the mouse then all your sockets would drop. Fun times.

> In MacOS if you grabbed a window title bar for too long with the mouse then all your sockets would drop. Fun times.

I wonder if the people developing windowing systems with client side decorations (e.g. Wayland + Qt) thought about this issue. Any information about that?

As far as I'm aware there's two approaches (often both used at once) in modern GUI systems that mitigate this. One is an explicit event loop, which can be locked up by user code but not normally by routine events, and the other approach is running the UI on its own thread. (Or, to put it another way, applications should perform any long-running operations not on the UI thread.) The second, particularly, would avoid the dropping-sockets problem, as network and drawing would very likely be on separate threads.

Erlang has preemption and process isolation in user-space.

Go's goroutines also somewhat approach proper user-space preemption/isolation.

Yeah, it's not really the user-space part I have an issue with, just the lack of preemption. Erlang obviously has concurrency solved pretty well.

Monadic futures are a fairly effective solution to the problem.

No callback hell, just a clean linear flow of imperative looking non-blocking code that runs concurrently but still guarantees order of execution.

Haskell is nothing but indirect action (lazy evaluation...), which is why referential transparency is so important to the language.

In my experience referential transparency increases simplicity and readability in most languages.

Yes, but so does keeping your code first order, avoiding callback functions even without the hazard of side effects.

I don't want to apologize for the existing callback-oriented APIs, because I agree it's easy to create some pretty hard-to-follow code in them. But there's something to be said for using non-blocking I/O calls. Threads aren't free, and spinning up a lot of them can tax system resources and impede performance. Allowing a number of execution contexts to share a single thread, automatically relinquishing control when they initiate an operation whose result won't be available for a while, can be a more efficient approach in certain circumstances.

Given that, I do respect what Microsoft was trying to do with async/await in C#. I'm not convinced they were entirely successful, though. The sales pitch is that you can happily bang out asynchronous coroutines as if they were just sequential code, but the truth is that if you actually try to do that you're liable to run into some nasty bugs.

I'm somewhat more intrigued by what some functional languages have done with the dreaded M-word to attack the same problem. The sales pitch is similar, but it's rather easier to deliver on the promise when "Just code like you normally do" being directed at someone who's normally writing in a functional rather than imperative style.

Python has achieved pretty much the same callback hell avoidance, without the need for monads or a foray into functional programming (in an imperative language). Python does it using generators as preemption points and using syntactic sugar for generator composition (yield from).

The code looks rather nice, for async stuff: http://www.getoffmalawn.com/blog/playing-with-asyncio

Using generators is rather clever. It's a look on generators not as fancy iterators that generate values but as fancy iterators that generate code execution steps.

Does it avoid the "Oops I accidentally forked a bunch of stuff I needed to run sequentially off into parallel threads that will now execute in an indeterminate order" situation? That's probably the biggest problem I see with C#'s async/await. Lovely syntax alone isn't sufficient, you need lovely syntax that doesn't make it it easier to write buggy code with the syntax than without. I suspect, though, (but cannot prove) that the risk of callback tangling an inevitable consequence of trying to slap this kind of abstraction on top of imperative programming style.

The reason why I think success is more likely when functional languages try to do the same thing is the functional style itself: Explicitly passing your state around in arguments and return values rather than relying on side effects means that whenever you do need there to be a strict ordering among your callbacks, that will naturally express itself in the code.

That's not an uncommon opinion at all, and I share the same view as you. New languages/run times that leak their concurrency model to application code are ugly. I've sworn off Node, and all event driven run times, for this reason. Erlang is far superior in this regard. The VM has an amazing scheduler. Immutability and message passing are the icing on the cake that make writing performant and maintainable systems that scale a joy.

> Remember Windows 3.1 or old Mac OS with it's cooperative multitasking where you had to explicitly yield or risk freezing up the entire computer? It's basically the same thing reinvented with a nicer brand.

A thread deadlock can still freeze up your entire process if it held the wrong mutex. Multiple processes for hardware concurrency in a web server context makes way more sense. Want hardware concurrency? Just spin up multiple processes. You're still taking advantage of preemptive multitasking, while getting all the benefits of not worrying about it within a single process's context.

> We came up with preemptive multitasking for a reason: because it's more robust and efficient.

For coarse tasks, sure. Throw a few thousand OS threads at a problem and you've exhausted your entire 32-bit address space on stacks before doing anything useful. It will not perform well in a 64-bit address space, although at least it won't crash. Probably.

Traditionally the answer has been to turn the task into a state machine, callback spaghetti, or whatever else. I find these answers to be quite lacking vs e.g. .NET's user-space async stuff, which I can use alongside threads if I so desire (as I frequently do.)

> Threads are dangerous, but basically you can avoid the danger for the most part if you avoid sharing objects across threads

This defeats the entire point of using threads instead of processes.

I challenge you to name a single advantage that threads have over processes beyond ease of sharing data. I imagine that such an advantage exists, but I cannot think of one myself.

On modern OS's, don't threads generally start up quicker and take less memory than processes? My understanding was that was actually the original motivation for the invention of threads, making something lighter-weight than an OS process.

But I think it's true that programs generally share some data between threads; but best to remember to keep the shared data as minimal as possible, and/or limited to immutable data structures.

> On modern OS's, don't threads generally start up quicker and take less memory than processes?

Yes... but the thing is, that's largely as a function of sharing data!

The OS can skip image relocation for the executable and it's dynamic libraries by sharing the same data with the other threads of the process. Process metadata is shared, so no new process entries need be created.

Your program can skip the parsing of configuration files, command line arguments, and other initialization by sharing the same data already parsed read and initialized by the other threads of the process.

On some platforms, there's fork(), which lets you retain some of the advantages of shared data. Not all of them: For example, one of the tradeoffs of a compacting garbage collector is that the relocation of data can cause those pages to become no longer shared between the forked processes, even if the data on those pages was otherwise untouched and identical, because the compacting isn't shared between processes.

The general trick is to keep a pool of processes ready to handle work, much as efficient thread pools avoid spinning up new threads for every tasklet.

But every async platform I know of has a non-async way of handling long running tasks so you can background them. The async model is good for responding to things in a quick fashion, such as http calls to a simple API. If the process you want to accomplish takes a long time the standard operating procedure is to background it and send a callback for (possibly to another api) when it's completed.

And yes the threading model works, on one machine. But across machines or OS instances? For that you'll need some kind of mutex syncing and you're basically back in the same boat you were in with the methods mentioned above. So why not use a platform specifically tailored to multi-instance clusters and save yourself some hassle?

Whether or not you can scale across machines has nothing to do with the thread model you choose. It's just about whether or not your server is stateless across requests, but you can do that with both async and non-async servers.

Yeah but additionally people are writing their source code by hand in CPS which is really stupid.

Reply on news.ycombinator.com