Concurrency is the New Memory Management
arschles.github.io
arschles.github.io
Remember Windows 3.1 or old Mac OS with it's cooperative multitasking where you had to explicitly yield or risk freezing up the entire computer? It's basically the same thing reinvented with a nicer brand. I know I'm dating myself here, but it's really the same model. We came up with preemptive multitasking for a reason: because it's more robust and efficient.
Threads are dangerous, but basically you can avoid the danger for the most part if you avoid sharing objects across threads, and if you do that then with blocking IO your code flows linearly and you don't have to deal with callback hell.
Go's goroutines also somewhat approach proper user-space preemption/isolation.
In MacOS if you grabbed a window title bar for too long with the mouse then all your sockets would drop. Fun times.
I wonder if the people developing windowing systems with client side decorations (e.g. Wayland + Qt) thought about this issue. Any information about that?
Not at all. First, don't confuse preemptive scheduling with time-slice scheduling. All "true" user mode threads are preemptive (Erlang, Go, Quasar). Not all of them employ time-sliced preemption. In fact, when we implemented Quasar on the JVM we had time sharing but then took it out because it didn't gain us anything (other than increased implementation complexity). The reason is that fibers in interactive applications follow a certain pattern that always entails frequent blocking. You are right, though, that some types of computations do not fit well with the user-mode threading model -- long, CPU intensive computations. Those are best left for kernel threads that do time sharing well. Languages that don't give you access to kernel threads might implement time sharing for their lightweight threads in user mode (e.g. Erlang).
Also, erlang doesn't use time slices, it counts "reductions" (essentially function calls, though I don't think it's always exactly one function call). It also penalizes processes more if they send to a mailbox that is already very large.
Erlang is considered truly pre-emptive, because (assuming you aren't writing your own functions in C or something) a function can't loop or use any operators or really do anything without potentially being pre-empted.
Go is generally considered partially pre-emptive, because it only pre-empts on a function call or during a memory allocation. You can write a simple loop that doesn't terminate, and it will never get pre-empted.
Well, in Quasar we started taking the "fully preemptive" route, but we saw that threads fall into two categories: those that block very often, and those that don't. Because the JVM, unlike Erlang and Go, gives you access to kernel threads, too, Quasar will simply warn you if you're using a fiber for a CPU-intensive operation that doesn't block often. Using reductions didn't work out so well because a "forcefully preempted" fiber still wants more CPU, which normally means it's doing something wrong.
Erlang is designed for reliability, not CPU efficiency for CPU-intensive tasks.
It's not a corner case -- it means any call to a C library needs to be carefully vetted so as to not cause an overly long computation or blocking because just a single one will totally ruin your concurrent experience. ALL VM:s using user mode threads/fibers suffer from that problem.
Real kernel threads doesn't have a lot of overhead and are almost always a much better choice.
Yes, and that's the definition of preemptive. There's nothing explicit about it if the VM handles it for you. What you're saying is that it cannot preempt kernel-thread blocking code.
> Real kernel threads doesn't have a lot of overhead and are almost always a much better choice.
Well, as usual, that depends. A kernel thread task-switch is 20-50us, while a fiber task-switch is a couple nanos at most; that's a 1-2 orders of magnitude difference.
Also can you please find a reference that shows (Linux) kernel threads being 1-2 orders slower than user threads?
And yes the threading model works, on one machine. But across machines or OS instances? For that you'll need some kind of mutex syncing and you're basically back in the same boat you were in with the methods mentioned above. So why not use a platform specifically tailored to multi-instance clusters and save yourself some hassle?
Given that, I do respect what Microsoft was trying to do with async/await in C#. I'm not convinced they were entirely successful, though. The sales pitch is that you can happily bang out asynchronous coroutines as if they were just sequential code, but the truth is that if you actually try to do that you're liable to run into some nasty bugs.
I'm somewhat more intrigued by what some functional languages have done with the dreaded M-word to attack the same problem. The sales pitch is similar, but it's rather easier to deliver on the promise when "Just code like you normally do" being directed at someone who's normally writing in a functional rather than imperative style.
The code looks rather nice, for async stuff: http://www.getoffmalawn.com/blog/playing-with-asyncio
Using generators is rather clever. It's a look on generators not as fancy iterators that generate values but as fancy iterators that generate code execution steps.
The reason why I think success is more likely when functional languages try to do the same thing is the functional style itself: Explicitly passing your state around in arguments and return values rather than relying on side effects means that whenever you do need there to be a strict ordering among your callbacks, that will naturally express itself in the code.
No callback hell, just a clean linear flow of imperative looking non-blocking code that runs concurrently but still guarantees order of execution.
A thread deadlock can still freeze up your entire process if it held the wrong mutex. Multiple processes for hardware concurrency in a web server context makes way more sense. Want hardware concurrency? Just spin up multiple processes. You're still taking advantage of preemptive multitasking, while getting all the benefits of not worrying about it within a single process's context.
> We came up with preemptive multitasking for a reason: because it's more robust and efficient.
For coarse tasks, sure. Throw a few thousand OS threads at a problem and you've exhausted your entire 32-bit address space on stacks before doing anything useful. It will not perform well in a 64-bit address space, although at least it won't crash. Probably.
Traditionally the answer has been to turn the task into a state machine, callback spaghetti, or whatever else. I find these answers to be quite lacking vs e.g. .NET's user-space async stuff, which I can use alongside threads if I so desire (as I frequently do.)
> Threads are dangerous, but basically you can avoid the danger for the most part if you avoid sharing objects across threads
This defeats the entire point of using threads instead of processes.
I challenge you to name a single advantage that threads have over processes beyond ease of sharing data. I imagine that such an advantage exists, but I cannot think of one myself.
But I think it's true that programs generally share some data between threads; but best to remember to keep the shared data as minimal as possible, and/or limited to immutable data structures.
Yes... but the thing is, that's largely as a function of sharing data!
The OS can skip image relocation for the executable and it's dynamic libraries by sharing the same data with the other threads of the process. Process metadata is shared, so no new process entries need be created.
Your program can skip the parsing of configuration files, command line arguments, and other initialization by sharing the same data already parsed read and initialized by the other threads of the process.
On some platforms, there's fork(), which lets you retain some of the advantages of shared data. Not all of them: For example, one of the tradeoffs of a compacting garbage collector is that the relocation of data can cause those pages to become no longer shared between the forked processes, even if the data on those pages was otherwise untouched and identical, because the compacting isn't shared between processes.
The general trick is to keep a pool of processes ready to handle work, much as efficient thread pools avoid spinning up new threads for every tasklet.
I say "kinda" related because even TM is a lower-level thing than the author of this post is talking about. The treatment of the subject, however, is much more in-depth. Lambda the Ultimate discussion: http://lambda-the-ultimate.org/node/2990
It is still in the small program stage although Glitch as a C# framework is usable now (indeed, I've written an editor, UI, and so on, all very concurrent). As a framework, it is not really convenient without being a new language, but it could be interesting to do something like ReactJS using replay and rollback.
If you like Bret Victor-style demos, I'm working on a web essay that should be done sometime this summer.
For concurrency in the large, I wonder if there will be greater adoption of these sorts of engineering techniques.
What does Go offer for a cluster environment? As far as I can tell, it's still targeted at single machines (though perhaps with many cores).
Which is another reason it might change how you think about architecture. ZeroMQ makes it practical to bring in other new languages and libraries into your ecosystem.
I'll have to disagree about the video games bit. Today's consoles are a completely regular AMD x64 CPU together with a completely regular AMD GPU, one running a version of Windows, plus one that uses PowerPC.
Yesterday's were two with multi-core PowerPC and an AMD GPU, one running a version of Windows, plus one with that really weird PowerPC together with 7 tiny PowerPClets and an nVidia GPU that was famously awful to program for.
Going back even further, we have an Intel x86 together with an nVidia GPU, running a version of Windows, a PowerPC-based CPU and an ATI-developed GPU (minor note: the company, ArtX, that was doing the GPU got bought by ATI and their designs did end up in ATI GPUs, so I'm listing it as such), a Hitachi SuperH CPU together with a PowerVR GPU, running a version of Windows, and this weird design from SONY.
And going back a further iteration things get even weirder and so on and so forth.
I'll agree that the further we go, the more these systems are made from specialised components (hello Saturn) and don't resemble what you'd get from e.g. a Dell. But that is mostly because you needed to use specialised components to get really cutting-edge performance for those tasks. Today you can more or less stick in 4 x64 cores and forget about it.
That only works for embarrassingly parallel tasks. Start requiring coordination, and today's hardware makes efficient parallelism plus concurrency hard.
A systems architect with a sound background in mathematics may be able to reason out the cluster performance of a social algorithm operating on a Erdos-Renyi random graph. Plodders such as myself will try to prototype. I'll insist that a prototype has business value if only because otherwise I have no way of appearing to be busy.
For prototyping a distributed system on a commodity cluster, my personal preference would be Erlang. As rvirding commented with great insight in another post, Erlang has an OS feel to it. From my limited perspective, Erlang spares me the trouble of knowing Unix and networking (for example: I don't need to know what a TCP port is). Erlang gives me a minimal & consistent set of verbs, and that's all I need for prototyping.
I use Go and I use Erlang. I've never used Akka.
Go is not in the same class as Erlang when it comes to Cloud Computing. In Erlang the state and behavior of your entire datacenter can be contained in Erlang itself. Erlang will take care of bringing up new processes when your jobs break. Erlang will handle messaging between processes executing on different machines.
Go doesn't do any of that out of the box. There are some interesting projects like Go Circuit, which sort of kinda want to emulate Erlang's OTP and bring your datacenter behavior into Go, but Go Circuit isn't being run in production by anyone I have heard of. Maybe its ready to go, but it looks more like a research project to me.
As such, Go has nothing to do with managing a cluster other than that it gives you some nice single node concurrency features. You will need to build or import everything else you need.
I can't think of anything else out there which is quite like Erlang/OTP. Maybe Julia has some tools that are similar but I don't know too much about it.
Edit: For those Go programmers that are interested in what I'm talking about, check out this concise explanation from the Erlang docs: http://www.erlang.org/doc/getting_started/conc_prog.html
Erlang isn't as user friendly as Go, but it has a lot of stuff that can save you serious headaches if you think you will need to scale. Premature optimization is a waste of time ... blah blah etc., so maybe you don't need it for your project. Also Go on App Engine allegedly scales pretty well with very little programmer effort.
- non-blocking I/O - user-space concurrency - easy messaging - advanced scheduling
From those, only easy messaging actually helps creating clusters, and it can be implemented in a library without any problem.
The other may help the language being good, or help one get more throughput from a node, but offer no help in maintaining the consistency of a cluster.
Erlang is really, really old-school actually - 1986. The language of the day before yesterday.
Alternatively there is also F# with it's Erlang-inspired http://en.wikibooks.org/wiki/F_Sharp_Programming/MailboxProc...
Most languages have extremely poor support for separating interactions with the real world from interactions that occur entirely within one serializable context (a thread working with thread-local memory, for example). Can you safely replay arbitrary C, C++, etc.? No, because side-effecting code could run at any time and occur in any context. So that is one problem that has to be solved first.
Suppose you've solved that problem. Well, now for a distributed system, you need to make STM talk to STM. Make one module talk to another over a network, or a filesystem, or another database, or a client browser. Do you have STM working in the JavaScript running on client's machines? And even if you managed that feat, do you have end-to-end STM from your datastore to your client's actions?
Distributed STM that wasn't painful to use, either to write or in terms of performance, would be a sort of holy-grail of distributed computing. I don't think any language or toolchain is there yet.
[1] Haskell, Clojure, et al STM engines.
[2] SQL-compliant relational databases such as DB2, Oracle, SQL Server, as well as distributed databases like HyperDex that support true transactions.
[3] Paxos, Raft, and other decision-making algorithms only ever externally appear to be consistent, but are internally complex and might have an internal tug-of-war.
I spend my time solving this problem. You can do it with a programming model especially designed for it. Functional programming is not required, though can be convenient.
> Well, now for a distributed system, you need to make STM talk to STM. Make one module talk to another over a network, or a filesystem, or another database, or a client browser. Do you have STM working in the JavaScript running on client's machines? And even if you managed that feat, do you have end-to-end STM from your datastore to your client's actions?
STM is the wrong way of thinking about this problem, mainly because replay is not an intrinsic part of the paradigm (instead, users manage that themselves). Rather, go back further to Jefferson's virtual time/Time Warp system [1], which was designed specifically in the context of distributed systems.
[1] http://dl.acm.org/citation.cfm?id=3988
There are also many related systems like Concurrent Revisions, LVars, Bloom, and so on...
With few restrictions, it's actually possible to replay arbitrary assembly. Namely, don't care about timing and no system calls that can't be replayed and you basically have what "checkpoint" does in gdb.