Are there any languages that have really nailed this? I've used gevent, eventlet, (both python), promises, callbacks (node) and none of them come close to being as productive as synchronous code.
I'd like to try out Akka and Elixer in the future.
I'd like to try out Akka and Elixer in the future.
Part of the problem with Python ecosystem is the insular mind set of its proponents. Python fanboys have no interest in going and seeing whats on the other side. So the platform has become a bit of an echo chamber with Pythonistas declaring their clunky approaches the industry best.
You can see this by looking at how little love a CSP solution for python gets [https://github.com/futurecore/python-csp] verses the enormous buy-in it's more popular frameworks receive.
Maybe there is a lock to implement the thread macro (clojure only), but then that uses native threads. How would you propose to handle access to channels between native threads without locks?
As far as I know there is no locking performed in asynchronous code implemented using the go macro. The go macro is a macro that turns your code inside out into a state machine, is it not? Each <! and >! point becomes and entry/exit into that state machine. There are no locks here because the go macro can essentially "rewrite" your source code for you and there is only a single thread of execution through the interconnected state machines.
I like to point at Facebook's use of Haskell as a good example of being successful in this space http://community.haskell.org/~simonmar/papers/haxl-icfp14.pd... It would be disingenuous to suggest that Haskell is good in all situations, but if there was one place where it should be used, this it.
¯\_(ツ)_/¯
The second is that do-notation is distinct from the IO monad. Even if Haskell didn't have green threads in the runtime, I could still write an async/callback library that looked just as natural as sequential code. Why? It has nothing to do with the IO monad: it has to do with the fact that "do x <- e; m" desugars to, in JavaScript notation, bind(e, function(x) { m }); it's been "callbackified automatically".
You can do that in any language with AST transforms powerful enough to transform code into CPS. That functions are "automatically callbackified" is an implementation detail and not one particularly germane to high-level code.
The IO monad enforces sequence on IO operations, and when you fork it, you get a new, independent sequence of IO operations to play with, not a new thread.
Haskell is really great for concurrent programming. Not only because of green threads (the mainstream concept that is nearest to the IO monad), but because of the "everything is immutable" rule, and very powerful primitives available.
It's actually much subtler than that due to lazy evaluation. The 'palindrome' section of http://learnyouahaskell.com/input-and-output illustrates this.
You basically write code that says: read all input, split lines, do something per line, join result into multiline string, print.
If you did that in Python it would block until EOF on the input and then print the result. In Haskell, due to lazy evaluation you will interactively get the response on a per-line basis.
What Haskell really gives you is a declarative language to describe what you want to happen to some data and then the lazy evaluation sorts everything out. Underneath, it's effectively a load of callbacks processing data streams.
It's a bit like Excel. Every cell is a variable that contains an expression, which defines what this cell evaluates to. With that description in hand, it's a simple matter of not evaluating cells that are not in view, and marking an exception in the evaluation with #######. If it were Python, each cell could contain code that modifies other cells, and it would be impossible to make sense of anything.
The biggest reason for this is not that necessarily that I think it has absolutely the best concurrency model, but that it's the most consistent one. Nearly all libraries are written for the model, which means they assume multithreaded access, blocking IO (reads/writes) and no callbacks. As a result most libraries are interoperable without problems.
Erlang/Elixir should have similar properties - however I haven't used it.
Javascript has a similar property because at least everything assumes the singlethreaded environment and concurrency through callbacks (or abstraction of them like promises and async/await on promises). I also like the interoperability and predictability here. But sometimes nested callbacks (even with promises) lead to quite a big of ugly code. And calling "async methods" is not possible from "sync methods" without converting them to async first (which could mean some big refactoring). So I prefer the Go style in general.
The worst thing from my point of view are all the languages that do not have a standard concurrency model, e.g. C++, Java, C#, and according to this article also Python. Most of them have several libraries for (async) IO which can be beautiful by themselves but won't integrate into remaining parts of the application without lots of glue code. E.g. boost asio is nice, but you need a thread with an EventLoop. If your main thread is already built around QT/gtk you now need another thread and then have 2 eventloops which need to interact. Some question for Java frameworks, e.g. integrating a Netty EventLoop in another environment (Android, ...). In these languages we then often get libraries which are not generic for the whole language but specific to a parent IO library (works with asio, works with asyncio, ...) and thereby some fragmented ecosystems.
A standard question that also always arises in these "mixed-threaded" languages when you have an API which takes a callback is: From which thread will this callback be invoked? And if I cancel the operation from a thread, will it guarantee that the callback is not invoked. If you don't think about these you are often already in bug/race-condition land.
I never understood why people tout Go's goroutine feature so much. You can have it in literally any systems language.
It's like saying that indoor plumbing is no big deal-- it's just liquid moving through a pipe. Well yes. Yes, it is. But if you don't have plumbing in your neighborhood, or a sewage treatment plant in your city, you can't fake it by fooling around in your garage. And frankly, it's not going to smell like a rose.
[1] https://github.com/btrask/libasync [2] https://github.com/btrask/stronglink/blob/master/res/async_s... [3] https://github.com/btrask/libasync/blob/master/src/async.h#L...
Disclaimer: not production ready, for most values of "production"
Edit: stacks don't grow dynamically, of course. But that's also a problem in Go if you want to efficiently call C libraries. If you really need efficiency, you can use raw callbacks for that particular section.
Of course, this also means that Go is making it hard for its libraries to be used by other languages. So it's probably a bad candidate to write something like a cross-plat UI toolkit, if you hope for its wide use.
In contrast, threads and callbacks are both well-supported in existing languages; so if you write a library in C using either, pretty much any language will be able to consume it.
In general, "turducken" designs are awkward and difficult to debug. Ask someone what a joy debugging or writing JNI or CPython code is some time. People often prefer "pure" libraries even when the performance is a little worse. C is the king of libraries awkwardly jammed into existing programming languages, but it's a dubious crown to have. Rust is trying to break into this space, but I'm not sure whether it's really a space worth being in.
So yes... I guess it is a pretty good analogy for C++. Hundreds of stinky septic tanks, right near hundreds of wells, along with a guy arguing indoor plumbing is overrated. And a buggy, informally specified reimplementation of half of Common Lisp ;)
I never understood why people tout Go's goroutine feature so much. You can have it in literally any systems language.
There are two big reasons for it.
Firstly, goroutines are extremely lightweight. "Traditional" threading in C, C++, and Java means native OS threads, which are comparatively expensive. Sure, fiber/coroutine libraries exist for these languages, but they are far from common (and, the only fiber library for Java that I know of, Quasar, came after Go).
Secondly, Go's ecosystem encourages CSP-style message-passing, rather than "traditional" memory-sharing. This is channels, not goroutines, but they make working with goroutines very nice. This is less concrete than the first reason; you certainly can implement message-passing in any of the other languages' threading styles. But empirically, it doesn't happen as often. A factor in this is also that, unfortunately, many CS curricula don't discuss CSP, which means that Go's use of this is the first exposure many programmers have to it.
It's sad that people use choice-of-language as a proxy for choice-of-execution-strategy (interpreted? JITed?), choice-of-allocation-strategy, choice-of-linking-strategy, choice-of-packaging, and so on. All of these factors should be orthogonal. By linking them, we create a lot of inefficiency by fragmenting our efforts.
AFAICT, C++ is the only language that's really been successful at being multi-paradigm.
Languages have to hand you very strong default choices for those things, because only the people with the hardest problems and the most time to solve them can afford to pick up a toolbox-box and build their own toolbox to solve a problem. Even the languages that arguably want to be that low of a level like Rust or D still have to offer a much more batteries-included standard library that will make more of those choices for you, and which will be for the vast majority of users the "real" version of that language.
Aside from that, personally I've used both Akka and plain Scala with Futures, as well as node with Promises, bare callbacks and async (though I've not tried fibers). I find Promises and Futures are the perfect balance between simplicity of use and the benefits of using the Async model. There's no need to reason about threads, as they abstract away the actual async implementation, and the interface they expose is very easy to reason about.
From the perspective of uniformity and availability, while C# provided asynchronicity via callbacks before the introduction of Tasks in the 4.5 release of the .NET Framework, all the core libraries that used callback-style async (as well as some that had been strictly synchronous-only) were updated with Task-based overloads, so there are no problems with Task-based async being inconsistently available. Additionally, adoption of Task-based async in third-party libraries has been high, so it's relatively uncommon to encounter code that does not support it.
From the perspective of code productivity, it's hard to get much better than simply adding the async and await keywords where necessary. As a very simple example, consider a typical server application that receives requests via HTTP, processes them via an HTTP call to another service as well as a database call, and then returns an HTTP response. The sync code (blocking with a thread-per-request model) might look something like this:
void handleRequest(HttpRequest request) {
var serviceResult = makeServiceCallForRequest(request);
var databaseResult = makeDatabaseCallForRequest(request);
sendResponse(constructResponse(request, serviceResult, databaseResult));
}
async Task handleRequestAsync(HttpRequest request) {
var serviceResult = await makeServiceCallForRequestAsync(request);
var databaseResult = await makeDatabaseCallForRequestAsync(request);
await sendResponseAsync(constructResponse(request, serviceResult, databaseResult));
}
async Task handleRequestAsync(HttpRequest request) {
var serviceResultTask = makeServiceCallForRequestAsync(request);
var databaseResultTask = makeDatabaseCallForRequestAsync(request);
await sendResponseAsync(constructResponse(request, await serviceResultTask, await databaseResultTask));
}
Lack of generics on channels really hurts the library ecosystem though. Many things you need to write yourself.