Fearless Concurrency: Clojure, Rust, Pony, Erlang and Dart
sites.google.com
sites.google.com
This is definitely a good high-level article, just something I was surprised by, since core.async is what drove me (and several other people I know) to start using Clojure.
EDIT: Just a note that I know core.async isn't built in, that was a mistake. It is a first-party library, however.
As for the general use case, atoms and core.async are great tools, and I haven't needed to use refs (aka the STM) for years.
In addition to this, I personally find core.async to deal poorly with flow control (e.g. slow subscribers slowing down the entire flow for every subscriber), and seems to not have a lot of telemetry / it’s difficult to find out what’s going on in the thread pool behind it.
I’ve personally settled on Zach Tellman’s Manifold library, which provides a much more sane abstraction and is fully compatible with core.async.
For some addition thought and material on these topics, I highly recommend watching this talk by Zach, “Everything Will Flow” https://youtu.be/1bNOO3xxMc0
Agents also have a fixed thread pool, and you should use send-off if you intend to do I/O.
This is also a bit of a departure for people coming from Go or Erlang/Elixir.
It seems like there are a lot of parallels between agents and core.async.
Agents, like go blocks, run in a thread pool. Any blocking IO should be pushed off to a separate thread outside of the thread pool. In the case of agents, there is (send-off). In the case of core.async, there is (thread)
I mentioned in an edit that I realized core.async is a library. It's still first-party, so it's still somewhat idiomatic.
major issues list memory leaks. https://dev.clojure.org/jira/secure/IssueNavigator.jspa?rese...
first-party?
By first-party, I mean it seems to be officially part of the Clojure ecosystem, the docs are hosted on the main site: https://clojure.github.io/core.async/
It doesn’t mean it’s not production ready.
* they really do need M:N threading (M green threads for the actors, N system threads where N ~= #CPUs),
* message queues have some complexities; it's not hard to get into a situation where messaging costs overwhelm actual work. Also, you need some form of back-pressure to keep a fast producer/slow consumer from killing everything.
Pony is actually a pretty sweet design for all of these. Complicated, though.
It's pretty trivial to construct actors out of some other message passing system. The interesting design choices in doing so are mostly down to the semantics in A) message passing, and B) scheduling/triggering/mapping-onto-green-threads/etc.
And for part A, all the other choices in the first three segments of the article are still effectively the choices that you've got to contend with: copying vs immutablity (bonus, COW mode, but still) vs ownership semantics.
Or to come at it from the other way: actors are far more featureful than is appropriate to directly compare to mere message passing semantics choices, because actors generally have some concept of error handling as a result of their relationship to scheduling, and that puts them on a whole different field.
This is something that is usually glossed over / afterthought when it is kind of a big deal in real life programming.
* Processes are extremely lightweight, orders of magnitude smaller than operating system processes or JVM threads.
* Exceptions should generally not be handled; instead, those lightweight processes are allowed to fail, and an external supervisor process will re-launch if appropriate.
* Assertions about the state of the data are effectively enabled on every line of code, so the processes crash early, before corrupting other parts of the system.
* Data is immutable, which plays a role in making those assertions happen.
You could probably strip one or two of those bullet points and/or add a couple of more, but I think that captures the highlights.
I'll toot my own horn. If you find that interesting, this is my favorite talk I've given about the above: https://youtu.be/E18shi1qIHU
[[someObject async] doStuff];
Yep, although not listed in the article, JS would be such an example for me. TBH I am not entirely sure what's the difference between Web Workers and Isolates but for me it's more or less the same: Both allow safe multi-threading simply because threads do have separate heaps.
> dart:isolate - This library was an attempt to provide a single API that provides common concurrency functionality across Dart's web and native platforms. While useful in some cases, most users found the isolate API limiting compared to the Web Workers API. The infrastructure for supporting isolates also adds substantial overhead when compiling to JavaScript. In the future, you should use Web Workers to access concurrency on the web.
> Part of the isolate API has never worked (spawnFunction) in JS platforms, and the other (spawnFile) doesn't work in Dart/AOT (Flutter). Neither are able to use transferrable/shared memory objects
> On the other hand, the JS platform has evolved quite a bit since isolates were created:
Web workers
Service workers
Animation worklets
... and other APIs like SharedArrayBuffer
... we'd like to enable our web users to seemingly use these, with as little overhead as possible
From https://groups.google.com/a/dartlang.org/forum/#!topic/misc/...
The power of channels could be illustrated better by adapting the next example to show how you can avoid shared state completely: https://gist.github.com/lorkki/e7386df8fff186b3e473994e9d31b...
It's also worth noting that a more versatile channel implementation than mpsc can be found in the crossbeam crate, in case you're thinking of using them for anything more serious.
https://en.wikipedia.org/wiki/SCOOP_(software)
https://www.eiffel.org/doc-file/solutions/eth-46802-01.pdf
It was ported to Java in a student project. That means other languages without great concurrency could probably use it with macros or a preprocessor.
TBF I have never written Pony code, I was mostly reading documentation.
Many languages don't have arithmetic operator precedence (Smalltalk, Lisp, etc) and that's fine.
It's not some feature that's missing, it's a design choice.
That's Lisp. In Smalltalk they do (have the usual infix syntax).
Isn't it the case that all operators in Smalltalk have the same precedence and left-to-right associativity? That's not "the usual" infix syntax.
Not to having "infix syntax plus the associativity / precedence you get in C". We've already established that it doesn't have the usual precedence.
The distinction was with e.g. Lisp which has a prefix syntax (polish notation), and other more exotic styles.
Does have a nice readline integration, though.
[1] https://github.com/sylvanc
[2] https://www.microsoft.com/en-us/research/people/syclebsc/
[3] https://qconlondon.com/speakers/sylvan-clebsch
Pony: Co-Designing a Type System and a Runtime [video] https://www.microsoft.com/en-us/research/video/pony-co-desig...
Jonathan Worthington can write and speak about this topic far better than I can so I refer you to his talk and slides:
1. http://www.jnthn.net/papers/2018-conc-par-8-ways.pdf 2.C https://www.youtube.com/watch?v=l2fSbOPeSQs
First the parrot threading model, which provided lockless safe threadpools, and second the pony threading model which provides the same on top of supporting shared refs, and forbids blocking IO. Which makes it even faster.
His talk ends with his simple approach taken being the best. Which is not only wrong but also a lie, because he was part in killing off the parrot threading model.
[1] shameless plug: https://github.com/duneroadrunner/SaferCPlusPlus#multithread...
Clojure has a full STM since version before version 1.0. Simple example:
(def stm (refs {}))
(alter! stm assoc :testkey "testvalue")
(println @stm)
Ref page 8-9 in [1]
[1] https://www.microsoft.com/en-us/research/publication/beautif...
Although, even a stray print inside an STM transaction could actually do you a lot of damage, since "print" is actually a fairly expensive operation. I've written all sorts of programs in my life that were technically bottlenecked not on reading the input, writing the output, or any of the processing in-between, but on the printing it was doing. And, relatedly, every community that tries to write a really blazingly fast web server in their language runs into a bit of a wall around the mere act of logging the hits. (Even just the date computation starts to bottleneck things, but the writing does too.)
You could of course argue that by preventing the user from doing certain things you avoid some classes of errors. However, I will in turn argue that by forcing the user to write code for the benefit of the type checker often results in convoluted solutions that are hard to understand and maintain. So, you just end up trading one set of problems for another.
Funny, this is exactly the opposite of my take from a type system like Haskell's. The thing is, whether it's enforced by types or not, the same invariants exist in your code. The only difference is that in one case they are checked explicitly at compile time, and the other case they are hidden and can blow up your programs.
If anything, explicit invariants make code easier to maintain and understand.
Static typing makes many dynamic patterns either difficult or impossible to use. For example, Ring middleware becomes pretty much impossible in a static language https://github.com/ring-clojure/ring/wiki/Middleware-Pattern...
The pattern here is that the request and response are expressed as maps. The request map is passed through a set of middleware functions, and each one can modify this map in some way.
A dynamic language makes it possible to write middleware libraries that know absolutely nothing about each other, and compose seamlessly. A static language would require you to provide a full description of every possible permutation of the request map, and every library would have to conform to it. This creates coupling because any time a library needs to create a new key that only it cares about, the global spec needs to be modified to support it.
My experience is that immutability plays a far bigger role than types in addressing the problem of maintainability. Immutability as the default makes it natural to structure applications using independent components. This indirectly helps with the problem of tracking types in large applications as well. You don't need to track types across your entire application, and you're able to do local reasoning within the scope of each component. Meanwhile, you make bigger components by composing smaller ones together, and you only need to know the types at the level of composition which is the public API for the components.
There is dialyzer, a static type (and not only) analysis tool for Erlang. It is part of Erlang/OTP (i. e. built-in.)
As a non-native English speaker I found this use of commas very difficult to understand. Often I get the feeling that native speakers don't even notice it.
The fourth comma is also incorrect and should be replaced by a semicolon.
"Unfortunately, the problem is that sometimes these tools are just not sufficient; it's still easy to shoot your own foot and get lost in a sea of complexity."
And looking at that now, I would also get rid of "the problem is that". "Unfortunately" and "sometimes" back to back lacks flow, so I would also replace "sometimes" (in this case with "often" in place of "just"). In the original, "just not" has a stronger grouping than "not sufficient", which is why "insufficient" wasn't used. Now with "just" gone, I'd swap back in "insufficient".
"Unfortunately, these tools are often insufficient; it's still easy to shoot your own foot and get lost in a sea of complexity."
"The problem is that sometimes these tools are just not sufficient. It's still easy to shoot yourself in the foot and get lost in a sea of complexity."
(Hemingwayapp is great for making sure writing is not too complex)
A tool that's "often insufficient" isn't a very good tool - by virtue of often being insufficient - it becomes [ed:simply] insufficient?
it would however be correct in German, so that might be it?
It has been supporting multiprocessor, multi-core, and multithreaded architectures for as long as it has been around. It has language constructs to make it really easy to develop, say, embedded parallel and real-time programs. It is such a breeze. I admit I am not quite sure what they are referring to by fearless, but if it means that they can handle concurrent programming safely and efficiently in a language, well, then Ada definitely has it.
Ada is successful in the domain of mission-critical software, which involves air traffic control systems, avionics, medical devices, railways, rockets, satellites, and so on.
Ada is one of the few programming languages to provide high-level operations to control and manipulate the registers and interrupts of input and output devices.
Ada has concurrency types, and its type system integrates concurrency (threading, parallelism) neatly. Protected types for data-based synchronization, and task types for concurrency. They of course can be unified through the use of interface inheritance, and so on.
If you are interested in building such programs, I recommend two books:
https://www.amazon.com/Building-Parallel-Embedded-Real-Time-...
https://www.amazon.com/Concurrent-Real-Time-Programming-Alan...
...other good resources:
https://en.wikibooks.org/wiki/Ada_Style_Guide/Concurrency
https://www.adacore.com/uploads/books/pdf/AdaCore-Tech-Cyber...
The last PDF will summarize in what ways Ada is awesome for:
- contract-based programming (with static analysis tools (formal verification, etc.))
- object-oriented programming
- concurrent programming
- systems programming
- real-time programming
- developing high-integrity systems
and a lot more. It also gives you a proper introduction to the language's features.
I agree that its open source ecosystem needs to grow, but for that we do need more Ada programmers! :P
By the way, I am really sorry if you experienced hostility from the community. May I ask where it took place? I had similar experiences with a variety of communities, even Rust. I try to not be demotivated from the language itself, after all, it is not particularly the language's fault, and there are people like that everywhere. They need to learn that what is obvious to them is not necessarily obvious to other people, and asking is a sign of wanting to learn, which I believe is a good thing. :)
As for where the hostility took place, it was most Ada proponents who would pop up in /r/programming, here on HN, etc. I'm sure the circumstances select for the most toxic folks from any community, but it seemed especially potent from Ada folks (could have been bad luck, ymmv and all that).
Would love for Ada to modernize and improve tooling/ecosystem, but between Go and Rust, I'm afraid that the advantages for a modern Ada might be marginal. It seems unfortunate for Ada that it didn't modernize prior to 2012; it could have eaten both Go and Rust's lunch before they even existed.
Assuming you know that none of the code in your concurrent blocks are is side-effecting, which may be hard to know if it's not your code; Haskell checks this for you, so to me it seems easier in haskell, not harder, because I don't have to hunt down and read all the source code to know if what I'm doing has concurrency bugs or not.
Take a look at Python's asyncio, which has been in the standard library for years, and you'll see how easy it is to write concurrent networking code without threads and with very very few uses of locks.
Trying to reason about how this could be possible. I'm still amazed by the performance you can get out of async non blocking style application code so don't take this the wrong way, truly trying to understand here.
I'm also a little confused about threads being rescheduled by the OS being treated like an inconvenience, to me it's a feature that prevents one application being too hoggish.
I can see your point that it's nice to have these labelled with await but I'm struggling to pinpoint a time I've asked myself what thread scheduling might mean for my code. Other than just assuming that another thread could be anywhere doing anything which I think is the correct approach no?
Locks btw, totally non issue with the right API/Lang support.
Naturally, if your application is computationally intensive, a single thread can't compete with a multithreaded application. But for applications that use a lot of slow, blocking I/O, converting them to a single-threaded application that uses non-blocking I/O is a significant reduction in overhead. Compare a traditional web server that uses one thread per request and blocking I/O, versus one that uses an event loop, non-blocking I/O. You will see why the latter is more efficiently on system resources. Again this isn't a panacea, and for some applications you do have to use threads. I'm just pointing out an omission of the article.
I'm not trying to debate on this, pretty sure we're just reasoning about facts we both agree on here.
I was aiming for a more general stance but if were talking about web servers there is one thing we might disagree on being the net benefit of non blocking in that specific scenario.
Let me see if I can explain, the individual requests may contain a lot of slow blocking io but these are handled async with a thread per request at the server level. So the request and internal blocking is limited to the request only (ignoring thread pool limits.)
While it may be that non blocking might produce more performant request level handling in some cases most or at least a substantial amount of web request logic is very dependent on chaining those blocking io one after the other, e.g. get thing modify, return.
In blocking you execute and reason about sequentially in non blocking you reason about with callbacks or syntactic sugar around promises specifically because sequential programming is easier to reason about.
The point of difference being that in non blocking you have a lot of overhead in the event loop and in the implementation, all to produce sequentially executing code that doesn't block other requests.
This is sort of why me and a few colleagues came to the conclusion PHP is still pretty damn decent for web stuff.
Non blocking is great for concurrency where it actually results in parallelism but if it doesn't then there isn't much gain other than gained complexity.
All this needs to be taken with a grain of salt. Node has Async await for a reason, and I've seen a few different implementations of non blocking php as well.
The choice really comes down to which style you find your self benefiting from on the regular.
Bleh that needs about three rounds of editing before it makes sense, something I don't have time for. Sorry for the ramble!
the pro is that your async code explicitly defines points where context switching is okay since you're blocking on something anyways. this could be good for perf if context switching in the middle of a synchronous operation is expensive.
the con is that your async code might not cede control often enough to allow other coroutines to make progress.
so yes, you can have something hogging the runtime but in the context of an application that you control as a whole this is something you can avoid/fix if necessary.
at the OS level this might not make sense because you have to assume that applications are adversarial and will try to hog cpu time...
> Notice that this article does not include Go, a language that admittedly has an elegant concurrency solution as well (Go channels) because that solution is not actually thread-safe - it's not very hard to have race conditions in Go or corrupt state because Go does not enforce a separation of shareable and not-shareable mutable state.
Generics will probably be added after the fact 10 years after the 1.0 release. They may not be quite as slick as something that was in there from the beginning, but based on the many other languages that added them after the fact, it'll probably work out well enough. Fearless concurrency can't be added to a language; if it's not in there from the beginning it'll never be added. It's a change so big it's almost automatically a new language. And, as can be logically deduced from that statement, I am aware that it would have some additional effects in the language, such as introducing immutability, it wouldn't be simply what we now know as "Go" with just a minor tweak. I'm comfortable with that. Even with Go's focus on simplicity, I do think there was a slight error towards being a bit simpler than the problem permits here.
(I do pretty well with Go's concurrency, but I was first trained brutally by Erlang and Haskell. I see those who don't come by the same route have more trouble.)
The example the author gives doesn't use them.
Fear, and respect, the absurdly difficult challenge that is writing correct concurrent code, even when your compiler is helping you out.
It's true that in the 90s it was important to cultivate a healthy fear of concurrency in the same way that parents in Cambodia must sadly teach their children to fear landmines. However, there's nothing inherent in the problem space that dictates that the concerns of one's ancestors must be the concerns of their descendants. One day we hope the landmines in Cambodia will be cleared, just as we hope the landmines in concurrent programming will be, and I'll be thankful when that day comes.
Personally I suspect generalized concurrency is always going to be something that "professional" programmers deal with, and non-professionals will only get pre-canned solutions for particular problems, because the general case will always be a bit more complicated than non-specialist programmers are going to want to handle. I think concurrency is worse in terms of what you need to keep in your head than raw pointers are, and those are already too much for a non-specialist.
That's what I mean by fear and respect for concurrent programming. That's the problem that hasn't been solved.
Also asynchronous systems can deadlock as well, it is just much harder to debug as the debugger won't show an obvious system thread blocked on some system call; the deadlocked threads of execution still exist but will have been subject to CPS and hidden from view (just some callback waiting forever on some wait queue).
Two processes blocking on a socket is not a deadlock. Surely there are timeouts on both sides, because using sockets without timeouts is just ignorance, and both will just timeout and move on.
Also strictly statically declared event handlers per actor are 100% mutual blocking and deadlock free. Because they can't wait for messages in a way, that blocks other event handlers.
Using timeouts to paper over issues is just wrong. I accept that timeouts are necessary to deal with network issues (and a timeout should cause the connection to be dropped, so won't solve the deadlock issue), but certainly they are not required for in-application message passing.
Finally if an actor won't send a message untill it has received another one, I fail to see how statically declared handlers will help.
Think of it as reacting to messages, not waiting. In that model actors of course can react by sending messages, but can't have a special waiting state for specific messages, making it impossible to block other handlers. I'm not sure why this is hard to understand.
We do have this problem solved in every possible way. But it's so not a big deal with actor model, that there is no point sacrificing any flexibility for it.
Edit: BTW, not sure why you got downvoted.
I mean, I'm no expert, but I guess you could statically analize the state machine and figure out, given a set of communicating actors, which sequence of messages would lead to a global state from which there is no progress. I assume that, because message ordering is not deterministic the analysis is probably non easy to do.
People have stopped using Java and Go?
There are plenty of safe and easy models for writing concurrent code. Here's a famous one that's easy to overlook:
gzip -cd compressed.gz | grep error
And of course, people have been writing shell pipes for decades without concurrency errors. Unix enforces process isolation, and makes sure all the data flows in one direction.
Now, there's no reason a programming language couldn't enforce similar restrictions. For example, I've spent the last few years at work writing highly concurrent Rust code, and I've never had a single case of a deadlock or memory corruption.
One slightly trickier problem in highly concurrent systems is error reporting. In the Unix example, either "gzip" or "grep" could fail. In the shell example, you could detect this by setting "set -o pipefail". In Erlang, you can use supervision trees. In Rust, sometimes you can use "crossbeam::scope" to automatically fail if any child thread fails. In other Rust code, or in Go, you might resort to reporting errors using a channel. And I've definitely seen channel-based code go subtly wrong—but not necessarily more wrong than single-threaded error-recovery code in C.
With the right abstractions, writing concurrent code doesn't require superhuman vigilance and perfection.
The halting problem does not need to be solved, since no program in practice runs forever. In some of the environments like C we don't have a way to specify finite things, where cases would be enumerable and safety guaranteed. To be fearless sometimes greatly reduces the mental workload of programmers and people who read other people's code.
Idris can already check for totality (i.e. guarantee its programs will halt) without solving the (unsolvable) halting problem. So we're almost there!
And even in Haskell, you can write certain classes of concurrent code without thinking thanks to 1) its great RTS and concurrency libraries and 2) the fact that Haskell programs compose so well: If I have thread safe programs A and B, Haskell gives me a variety of ways to compose their results in such a way that thread safety is always preserved.
As a long-time pthreads programmer, these systems feel like skittish concurrency. It's for folks who are too afraid to use the underlying concurrency primitives, like shared memory, synchronization of some kind provided by OS, threads.
I interpreted "fearless" to mean the opposite of "defensive programming." You have to be pathologically defensive when coding with basic synchronization primitives. These languages provide an alternative that frees you from that mindset. I only have experience with Erlang/OTP but it has been mind expanding when it comes to writing complex, robust, concurrent code. In fact one of the core Erlang style principles is "do not program defensively." Try that with pthreads!
And even if you think you never make mistakes, and it's always the other programmers who are the problem, Rust keeps other peoples' code in check too.
I've done a fair amount of pthread work, and I coded myself into enough hard-to-debug issues in C that when I discovered I could fairly-easily structure my stuff around ZeroMQ, and then later discovering a book on different process calculi, I decided that there's no way I can go back to doing the low-level stuff. I'm just not smart enough to handle any kind of elaborate threaded programming.
(Did the Python crowd ever fix the race condition in CPickle?[1] They were in denial about this about eight years ago when I reported it. Doing multiple CPickle operations in separate threads can crash CPython. If you search for "CPickle thread crash" you find many reports of hard to reproduce problems in that area.)
Correctly? What does anyone ever do correctly in software? Does anyone write GUIs correctly? Does anyone do databases correctly? Proving even simple software correct is an enormous endeavor that most pragmatic programmers simply never have the luxury to do. So proclaiming that threads are bad because nobody does them “correctly” is setting an arbitrarily high standard. Or maybe you just don’t know what that word means.
Fearless concurrency means guaranteeing many of those problems cannot occur. That frees up brain cycles, eases maintenance, reduces bugs, and perhaps even opens up concurrency to a wider audience or set of use cases.
I understand what the thing they call “fearless concurrency” is. And I will repeat again: it’s skittish concurrency. It’s for fearful people.