Comparing Haskell and Node concurrency performance
fpcomplete.com
fpcomplete.com
Perhaps my head has been stuck in javascript/node land for too long but I think accusations about javascript producing callback hell now seem a bit disingenuous even for relative novices to the language.
It's 2016 and there are many well documented and widely adopted solutions arising from external libraries and developments in ECMAScript. Thanks to transpilers like Babel/Typescript we can even shoehorn these new ECMAScript features into older browsers.
In the article we have someone promoting the use of a functional language on a functional programming site promoting the use of thread-lexical mutable variables to store state across procedure calls. The alternative the author dismisses is to swallow a return as an input. Something's not quite right here.
If you're doing something in Node that takes nontrivial processing time (that doesn't just involve waiting for I/O), you're Doing It Wrong (tm).
Since the premise of "callback hell" is outdated, the only remaining point is that if you perform complex/time consuming calculations in NodeJS, it slows down.
If you want the most speed, or if you're going to be doing something like a Fibonacci calculation, you should write that (microservice) in Go instead. It's faster from the start (by 2x at least, probably a lot more if you're CPU bound), and GoRoutines can be spread across multiple threads, so a multi-CPU host can take advantage of all of its CPUs.
Oh, and the per-GoRoutine overhead is only about 4K on a Linux host, last I checked. True that Haskell has a smaller overhead per thread, but considering the speed advantage, I think the Go server would still win big on latency for the number of threads that did actually fit in memory.
Granted they aren't all top quality packages, but thousands are.
I was working on one project that used ZeroMQ, as a random example. There are excellent packages for Node, including a well maintained version for the newest 4.0 branch.
For Elixir, when a team wanted to interface using ZeroMQ, the best they could find was a "work in progress, not ready for production use" build that was a partial re-implementation of ZeroMQ 3.1, and that specifically lacked the elliptic encryption feature that we were using. The more mature ports to Erlang similarly doesn't support the encryption we were using. [1] There's one native binding that seems to only support 3.1 (at most) as well [2], but it's something you need to build and configure, as opposed to "npm install zeromq --save", or better yet, "yarn add zeromq", and then your project will work on any system without any complex build rules to get it working.
That's just one package that I tried to use, and it's a popular network message queue package (two full ports to native Erlang!). If I had tried to do something more obscure I'm sure I would have had even more problems.
Another key advantage is that there are probably 10x the number of developers ready to hit the ground running on a Node project than there are Go developers, or 50x as many as Elixir or Erlang developers. I don't know if you've tried to do much hiring, but it's hard enough finding developers for a language that's popular. (And no, I'm not counting "front end" JavaScript developers; if I were, I would have said 100x or more.) If you're hiring in an area that isn't highly tech focused, you might not be able to hire a single developer with experience.
[1] https://github.com/zeromq/ezmq/issues/31 and https://github.com/chovencorp/chumak (supports only the "NULL" security framework -- there's a note lower down on the page that Curve isn't supported)
Only if you only plan to ever speak to other Erlang/Elixir hosts would it make sense to use an Erlang/Elixir-only solution. And I will not even consider the option that writing a game (which is what I'm doing) in Erlang or Elixir is reasonable. Yes, it's possible, in a "Turing Machine" kind of way. But no, it's a terrible idea. Complex games don't break down into isolated functional chunks without actually making them much harder to design, maintain, and extend. I use plenty of functional concepts when I write games, where appropriate, but using a functional language would be a nightmare.
Phoenix is a web framework. "More powerful" functionality that I can't use for a particular project is a pretty profound waste.
Since I did actually abandon ZeroMQ for my game (when I decided that I needed to do most of the game as a web client anyway, and so I should just make the host app be built on TypeScript as well), I'm now using Socket.io, which surprise has an outdated port to Erlang [3] that doesn't support 1.0, and there's no native Elixir port. See a pattern emerging?
And since I mentioned speed was important: Kami, a Go-based web framework, benchmarks up to 4x faster than Phoenix. [1] Raw Go speed for the kind of thing I'd actually be doing is closer to 5x-10x that of Phoenix. [2]
By the way, it turns out there's an actively maintained Socket.io port on Go. And being able to spin up 1/8 the number of servers could realistically take my project from break-even to profitable.
[0] http://zeromq.org/bindings:_start
[1] https://www.techempower.com/benchmarks/#section=data-r13&hw=...
[2] Look at the fasthttp Go results on https://www.techempower.com/benchmarks/#section=data-r13&hw=... and https://www.techempower.com/benchmarks/#section=data-r13&hw=... -- my usage pattern would be somewhere between raw text and "single db" access, since my DB is Redis on localhost, so it's more like a cache than a Postgres/MYSQL lookup.
Elixir and Erlang make writing concurrent applications easy, and they already have built-in mechanisms, which are better than ZeroMQ, for handling the type of communications that Node folks MUST turn to ZeroMQ to get. Using ZeroMQ in an Elixir application is like installing a gas tank into an electric car, just because they work so well in cars with combustion engines. Completely unnecessary.
Why would an Elixir programmer want to interface with ZeroMQ, when there are far better options for such communications built into the language itself? Node HAS to use ZeroMQ because it lacks the intrinsic ability to handle many problem spaces. Elixir and Erlang don't suffer from those shortcomings. In fact, they were designed to solve those types of problems. I've replaced ZeroMQ countless times when replacing Node code with Elixir code, and the end result has always been an amazing improvement in speed, stability, and scalability; not to mention code maintainability.
A lot of Node programmers tout npm as a great package manager, but how is it any better than gem, pip, and all the other package managers that it took its cues from? How is typing "npm install zeromq --save" any better or easier than typing "mix deps.get," "gem install zeromq," or "pip install zeromq?" Could it be that Node programmers are simply unaware than tools like npm have existed in other languages for years?
Finally, I'm sure there are 100X the number of Node programmers out there for every Erlang or Elixir programmer, but, when you need speed, stability, and scalability over trendiness, you really only need to have one.
So the lexical structure of my code was linear - while the runtime structure was nested at arbitrary levels. There never was a reason to represent the nested runtime structure in the written code.
I also didn't attempt to use node.js for things it wasn't made for, like compute-intensive tasks or implementing business logic. The good old chat server for ten thousand people was an often used example for node.js programming for a reason - lots of I/O, little processing.
Note: I don't write code like that any more in ES 2015. I also don't use classes, prototype, this, bind, apply - only functions and (lexical) scope (with an eye on capturing only as much scope as I need). Which is the opposite of the above described method where lexical scope was not usable, but with the methods available now the code still is "flat", so that's why I switched.
The key was of course to come up with great modularization, of course you would not want to do that with "flat code", the complexity of what function is where would be (or would have been, since I no longer need to write in that style) overwhelming.
That is, if you have the same nesting at runtime, but it is just somewhat obscured by the naming style that you did, that sounds problematic to me. Ideally, you find structural ways to get rid of that nesting. (I said elsewhere that I'm a huge fan of first class queues. There are other options. Callbacks are one. And realistically, what you describe is an option, too. None of them are intrinsically bad.)
Running performance tests with a single Node process doesn't feel like it's giving Node a fair chance to perform up to the capabilities of the machine.
"Solutions" is a strong word for what's available in JavaScript. Promise hell isn't better than callback hell, it's just horrible in slightly different situations. To make matters worse, it seems that many JS libraries have just wrapped the old callbacks in promises, meaning that we end up using promises in situations where a callback would actually be easier (because that's how it was originally written).
None of this comes close to the ease of threads in Erlang.
There are good reasons we went to thread based models - developer productivity and safety. Event loops are fine for toy demo's, or very carefully managed products (trading systems, NGINX) but not for use as general purpose hammers.
Every single bit of extra friction and cognitive overhead costs you dearly a few years down the line. We scrambled away from this stuff as soon as we could, and there's no good reason to go back.
"Callback hell" is IMO a terrible way of doing event-loop systems, as the nesting can get confusing. For implicit event loops like Node, I strongly prefer the Promises approach (preferably with async/await sugar); for explicit event loops (I only have practical experience with Arduino, though I've also a passing acquaintance with the classic 69k Mac as well) I like to build a set of event-driven state machines with cooperative multitasking. If properly designed, these keep everything nicely separated so you can follow the logic without any trouble, while also avoiding the concurrency concerns of a threaded system.
> However, Node v7 is based on V8 54, whose async/await implementation has some bugs in it (including a nasty memory leak). I would not recommend using --harmony-async-await in production until upgrading to V8 55; the easiest way to achieve that would be to wait for Node 8.
Also, I'd rather not use a non-LTS version in production since I don't want to have to keep track of changes. Most of the stuff I maintain sits untouched for months between releases; I think there might even be a few things still running on 0.10.
[1]: https://github.com/nodejs/promises/issues/4#issuecomment-254...
[1] https://github.com/tracker1/react-redux-materialui-boilerpla...
I greatly prefer first class models for where things queue up, and then to use producer/consumer objects against those queues.
I think producer/consumer and promises cover two different use cases (with some overlap). Indeed, when I have a work queue it frequently involves promises: queueing a work item returns a promise which will resolve when the work is complete, and part of executing a work item is returning a promise so the consumer knows when the task is done and can start in on another one.
Compared to knowing that you have a set of queues that ultimately feed into other queues. It is much clearer to reason about the throughput of individual queues and take that into consideration when designing new queues in the system.
It is basically like someone throwing a ton of outlets on a wire going through a room. I mean, yes. You can do that. Often won't even cause issues. However, for large enough systems, you ultimately need to know what the load on that circuit will be and it is not acceptable to put yet another plug extender there.
What pain awaits me?
It is also "bad software-engineering", as you are now forced to think about slicing up your code, which, in an already complicated application, can lead to bad code.
First, using clustering and memoization would improve the throughput a lot. I did something similar when adapting a JS based script library to be used in node, because I knew it would lock the main loop otherwise. Beyond this, cpu intensive work should be avoided in your service loop regardless. It's best distributed to an RPC/Worker pool.
In terms of scale, node scales as well or better than a lot of frameworks, it's only that you will usually want to use similar techniques locally as well as remote.
Another poor example is when you need millions of references in a single thread, Node will die spectacularly. That doesn't mean it shouldn't be used for many use cases, it only means that it's bad at some of them.
I find that node is great as an intermediate/translation layer... your UI talks directly to node, tightly coupled.. then node can translate against backend databases or other services as a gatekeeper for your front end. It allows you to make the data the shape that is most convenient, with the least amount of disconnect of thought and approach.
It's also pretty great for certain types of orchestration control and even in the proof of concept stages of applications. Doing a first version of almost anything I've tried in Node is usually much faster than alternative platforms. And often performs well enough to stick with it. Developer productivity is more important than absolute scale at the beginning, and if you have a plan to scale horizontally, you can do that for a while before you need to break off other optimizations.
Do you say this because of the libraries available for it? (curious, wanting to jump to node).
It's just so much faster to get going if you're developing the full stack, and already in JS heavy land anyway. Not having to context switch for the backend is huge... being able to use a document/object/json database isn't as big of a boost but still nice. On the db side, I've been using the template wrappers so that I can write a simple query and it turns it into a parameterized query returning a promise.
async function getRecords(baz) {
return await sql.query`
SELECT
a,
b
FROM
foo
WHERE
foo.bar = ${baz}
`;
}
Overall, for the past 6 years or so (since 0.8) I've been using Node pretty heavily (moving from more C# on the backend) and really haven't missed it at all. Even though the core .net and more open-source stuff has been interesting... Managed to dockerize a few trivial .Net apps using the dotnet onbuild base containers.
If you're using windows, the only gotchas are you need a C++ build environment (Visual C++ 2015 Build Tools, checking all options) and Python 2.7.x in order to build any binary modules... most of which now run without issue on windows, was a much bigger problem in 0.8-0.10 ...
Anyone who understands JavaScript can see that the recursion invoked in the slow route is not asynchronous (each recursive invocation keeps piling onto the call stack without ever releasing it)! You'd have to use process.nextTick (or setTimeout) if you wanted to recurse asynchronously without spawning a new process...
For these kinds of unusual, heavy computations, though, you'd be better off using the child_process module to spawn a new process and do the recursion inside that process so that it doesn't block the main event loop.
This has nothing to do with starvation. Node.js just has a completely different approach to this kind of problem.
GHC (the leading Haskell compiler) provides an extremely advanced green threading system with the runtime. http://haskell.cs.yale.edu/wp-content/uploads/2013/08/hask03...
> I like the way it's done in Node.js - Explicitly.
If you wanted to do it this way in Haskell, there are a number of monads that make it way more convenient than doing it in Node. Cont in particular can be used for cooperative concurrency. No one really uses these outside of niche cases, however, because threads are almost always a better abstraction.
But I do a lot of REST API (and WebSocket) work and all of the workload that happens inside my Node.js program is extremely lightweight.
If I need to perform some heavy computation, I will offload it to a separate child process - Node.js forces me to put that code in a separate file/module but I actually like this because it encourages separation of concerns. It feels very natural so I don't really need any other special constructs.
I looked into goroutines a while ago; it looks cool, but I probably wouldn't use them much because I don't like the idea of having code from the same source file splitting off into multiple processes/threads; it makes is harder to read and reason about the code (this is a bit like what happens with multi-threaded code when you have mutexes all over the place).
To me, this feature has the same utility value as the ability to define multiple classes per source file - Ok, that's cool, but is it a good idea to do that?
foo :: IO () -- optional type declaration
foo = forever (print "hi")
forkIO foo
https://hn.algolia.com/?query=why%20events%20are%20a%20bad%2...
https://hn.algolia.com/?query=why%20threads%20are%20a%20bad%...
https://hn.algolia.com/?query=threads%20vs%20events&sort=byP...
EDIT: Just to make it clear early on, I agree with the article's conclusion that Nodejs is not as good at compute heavy workloads as Haskell. I simply object to any use of "the nested callback problem" as valid in 2016. It's an issue exclusively for legacy code and developers who take pride in writing outdated code.
It seems only fair, then that I also should defend Javascript from people obviously unaware of the state of the art in pseudo-imperative programming. And by state of the art, I mean "has been around in some languages for 3+ years."
The example:
request('http://example.com/random-number', function(error, response1, body) {
request('http://example.com/random-number', function(error, response2, body) {
request('http://example.com/random-number', function(error, response3, body) {
...
});
});
});
// rp is a request promise, multiple options for creating them
async function make3StaticRequests() {
try {
var res1 = await rp('http://example.com/random-number')
var res2 = await rp('http://example.com/random-number')
var res3 = await rp('http://example.com/random-number')
// ...
}
catch(error) {
// ...
}
}
// And of course the promise library allows for many things
// you'd like with applicative functors, like binding groups
// of operations together and evaluating them all.
function randomNumberPromise() {
rp('http://example.com/random-number')
}
async function make3StaticRequests() {
var [res1, res2, res3] = Promise.all([randomNumberPromise(),
randomNumberPromise(),
randomNumberPromise()])
// ...
}
That said, the articles point about large compute workloads starving other operations is very much true and a good example of what the weakness of V8 as a server-side programming environment brings.
While Haskell is truly better at concurrency (no need to serialise when passing messages, green threads yield not only at IO but also at memory allocations), that part of the comparison isn't very good. Spawning a cluster of NUMCORES threads using the built in cluster module would be an improvement.
Nodejs has a poor story for compute-intensive loads. People need to be comfortable saying that, because it's reality.
Secondly, the service presented there doesn't even need to take advantage of shared memory concurrency at all. This is the case for a vast majority of web service problems too: they either talk a lot to each other or do a bunch of cpu-intensive work, but rarely both.
Finally, when you use shared memory concurrency/parallelism to solve web service problems, there is a risk that the resources of a single machine will not be enough. And then you are back to serialising things and sending them through an even slower channel.
Haskell also has poor facilities for compute-intensive stuff, although they are different facilities. For example, laziness makes reasoning about performance more difficult. Space leaks are fairly easy to create unless you know the common gotchas. Most naive/idiomatic Haskell code performs several orders of magnitude worse than whats possible with optimized code. Etc etc.
I disagree. Totally and utterly. API servers are webservers, and in fact a pretty big subgroup of them. API servers can often run into computational requirements in mid request, and it sucks that ONE slow request can cause latency ripples across your entire process.
> Haskell also has poor facilities for compute-intensive stuff, although they are different facilities. For example, laziness makes reasoning about performance more difficult.
This is changing the subject, and ultimately a non-sequitur. "This also has problems which are different" is not actually an answer to the criticism that nodejs is bad at these workloads.
Say it with me. It's okay to say. "Nodejs is bad at compute-intensive workloads."
Or it doesn't cause ripples, since the cluster master doesn't send requests to it, and instead redirects them to the other processes.
That, plus a process pool for known compute-intensive stuff is often good enough.
A methodology that doesn't scale well across boxes, so only buys you so much. Demo code might benefit from this approach, but production code will do something totally different to give an API or server actual durability and uptime.
> That, plus a process pool for known compute-intensive stuff is often good enough.
To make it "good enough" for production workloads for anything less than a trickle of traffic requires an entirely different and probably queue-based architecture. While this is often in good style, it is (I restate) a tool available to every language environment. If everyone has this technique, you cannot say that Node is given a pass because "if you just totally re-architect this code" then it's okay.
Nodejs's scheduler and execution model fundamentally make it worse at compute-heavy workloads. THIS IS AN ACCEPTABLE TRADEOFF. But denying it exists only misleads engineers and leads you to bad decisions.
Yes, you would have a load balancer in front of several instances running on several machines. Same as Haskell.
> Nodejs's scheduler and execution model fundamentally make it worse at compute-heavy workloads. THIS IS AN ACCEPTABLE TRADEOFF. But denying it exists only misleads engineers and leads you to bad decisions.
I agree, in principle. But one, its not nearly as bad as this article makes it. And additionally, the given example is not convincing or very representative.
If you do an internal N-M process fanout, will you also stop accepting connections on the individual box level if you're queueing? How tight are your health check timings to make sure this doesn't result in shitty performance when a pathological workload comes up? Will you disable the TCP connection queue here as well?
I would not. And I agree, it will be worse than Haskell, which will only choke if the cpu intensive processes are all calculating things without allocating memory or doing IO (as opposed to node where IO is the only yield point).
And if the article talked about when Haskell green threads yield and compared that with node, as well as measured just how worse things are with a clustered setup, with several example problems, and by varying the parameters (such as number of processes)... then that would've been really awesome.
With Node, you'd have to serialize data and pass it between child processes. And that really, really sucks. Haskell's parallelization story extends way past the "embarrassingly parallel" request handling.
A good data point based on a less contrived example is how handily Haskell web frameworks demolish Node.js at JSON serialization... and not much else (https://www.techempower.com/benchmarks/#section=data-r13&hw=...).
Finally, if you're looking to do something truly and extremely CPU-bound, I'd tell you to write it in a C derivative and bind it to Node or Haskell, regardless of what your favorite language is. Optimize for speed in some places, and programmer happiness in others. A one-size-fits-all approach isn't usually appropriate.
https://github.com/spion/fpco-article-examples
Now it gives the same results as the Haskell solution. (Which only means this concrete benchmark doesn't reflect real world performance)
That said, I agree that the article should at least mention the likes of `await`...
Begrudging Javascript 1 or 2 seems somewhat petty given how common it is to see compiler extension preambles at the top of every haskell module.
Haskell itself has no extensions, and you certainly don't need any extension to get what I was talking about above.
As a WebLogic developer we fixed this in the late 90s but the Volano chat benchmark was still run for no apparent reason. Somehow everyone else didn't get the message until Netty was released and people started using it.
Obviously what you want is multi-threaded execution with asynchronous I/O. Using node on multi-core systems just doesn't make a lot of sense as you end up having to duplicate your entire program on each core to get the full performance of the machine. Not unlike 512k/thread but much worse — especially if you cache anything locally in the process like template compilation, etc.
Don't get me wrong, the fact that Node doesn't have a more efficient way of handling overhead on multicore machines is a drawback. But you pay for your abstractions. Nobody is going to argue that Node is a phenomenal solution for lightweight, multi-threaded execution. But most companies using Node seem to accept this and are fine with not utilizing 100% of their resources (hell, most don't even seem to know Node can spawn processes, in my experience).
If they care they use a framework that meets their needs.
Tcl was doing event loops quite successfully in the late 90ies and was fairly popular, back in the day.
These days, I'd use Erlang (Elixir).
This is kind of ranty, but also makes me laugh: https://www.youtube.com/watch?v=bzkRVzciAZg
I don't know if you can claim definitively that Node introduced more people to event loops than any other technology, but it certainly is one major popularizer of the idea, and the one that's had the most impact in the past decade.
The fact that you knew it first doesn't give you a reason to be snide.
What would I call "heavy work"? e.g: compression, serialization, encryption, image processing... tasks that are bound by CPU and not only I/O. Usually you want to delegate that to a native module and not do that yourself in JavaScript. If you absolutely have to do it in JavaScript, then you need to make sure the task is not blocking the event loop. In order to play nicer with the event loop you inject something like setImmediate or process.nextTick after certain amount of time or iterations... otherwise you will starve other tasks in the loop, notably, I/O.
node is also not a really good idea if you need a lot of interprocess communication.
It is a very viable alternative, though.
https://github.com/spion/fpco-article-examples
Its an interesting benchmark, but it needs more work to give a more accurate picture. It would be nice if it:
* used wrk to measure requests and latency percentiles
* percentage of "slow" requests was tweakable
* number of workers per core was tweakable
> Looking near the top of the output, we see that Haskell's run-time system was able to create 100,000 threads while only using 165 megabytes of memory. We are roughly consuming 1.65 kilobytes per thread.
Those are not the same kind of threads that the author is talking about in the beginning of the article. Those a green threads and as such are multiplexed to much smaller amount of real system threads to do work in parallel. What that means is that they, for example, can't all make a system call at the same time. Go has the same issue.
Yes, they can. The Haskell IO manager knows how to handle this. http://haskell.cs.yale.edu/wp-content/uploads/2013/08/hask03...
The runtime will migrate pending green thread actions to other OS threads. I don't remember what happens if you have as many blocked FFI calls as OS threads. The runtime might spawn more if you're using RTS -N. If you have RTS -Nn, where n is a number, I imagine it does block.
This signal sending is done by setting up a periodic "timer signal" that sends SIGALRM to the process every 10 ms (by default).
giggles uncontrollably
I like Haskell, but saying that it doesn't impose a design burden is incredibly misleading.
node is probably not the best choice for truly CPU bound operations, but you can sometimes get by using the native cluster module to spread work over multiple cores.
With this statement, the author acknowledges that the Node.js code in the slow route was not asynchronous. The test is therefore invalid; it's comparing apples and oranges.
Node.js is more than capable of handling different requests asynchronously (regardless of whether they are fast or slow); if you have any kind of blocking or waiting around happening; then you're doing it wrong.
I'm so tired of all the anti-Node.js propaganda; it's hurting people. If I walk into one more company where some zombie tells me that they're migrating away from Node.js because "the Node.js event loop starves the CPU", I'm going to have a stroke.
In reality all Node.js 'starvation' problems can be solved with the 'cluster' module or the 'child_process' module.
Since we've been talking a lot about 'Fake news' on Facebook recently. Maybe we should start talking about how fake news is affecting Hacker News. This anti-Node.js strain is particularly virulent.
You can scale node, via the same techniques you use to scale anything across servers, you just do it sooner with node in order to better utilize a larger server, or use multiple servers sooner. Node is great for just about any io bound workflow.
1) The Author knows little about JS.
2) Picking on a scripting language which has 2 weeks + 2 years of development vs a 25 year old monster that is the playground of the brightest minds in CS, is easy.
Unfortunately the article did also not go into details of how the event loop works or shows how you can break out of the single thread with the tools that ARE available to you. I guess the idea was to write a hype article for Haskell. Here are some more ideas for the author:
Comparing Haskell and C++ type systems.
Comparing Haskell and Clojure functional purity and lazy evaluation.
Comparing Haskell and Java deterministic parallelism.
What might be more interesting:
- what is the salary difference between a Node dev vs. a Haskeller?
- is there a productivity difference? does the salary over- or under-compensate?
- is correctness a core business concern?
- if I need to hire 10 devs, can I do that?
Go watch the node.js presentation Ryan Dahl gave at JsConf 2009, he addresses this during that speech.
His opinion on the "right way" to do concurrency is a little polarizing, but, quote: "the right way to do concurrency is to use a single thread and have an event loop. this requires that what you 'do' outside of IO waits not take very long".