Node.js: Cluster vs. Async
synsem.com
synsem.com
- It implies that async execution is equivalent to callback hell. In reality there are excellent ways to have async code which looks just like sync (generators, async/await).
- It benchmarks multi-core (sync) vs single-core (async) and makes claims based on the results.
- It presents async execution as an antipode of clustering. In reality it's a best practice to make use of both.
...and everything that follows is just irrelevant.
I guess the author had to add the sort/reduce to "prove" his point...
> async cons: Adds latency with parallel overhead
What parallel? This should be cons for multi threaded running on single core cpu
> async pros: Callbacks help enforce error-prone synchronization
How does callbacks help in error prone synchronization? It is not callback, it is single threaded model that does.
Request per-process/thread has all the same memory overhead implication it has always had. It's almost like the author is ignorant of the reason for node.js' success or why it was built in the first place.
Also, "callback hell" is just FUD. Nobody who does this for a living and knows what they're doing really has an issue with this. Promises solve the unreliability issues, and async/await solves the syntax complexity issues.
I'd like to see this same analysis for 1000 req concurrency measuring memory overhead and using async/await for code comparison. Cooperative multitasking will always be capable of lower latency when you know what you're doing, and async programming is lightyears simpler than multi-threaded programming.
Launching that many threads will quickly hit bottlenecks on most systems... I've seen this happen in a poorly written simulation server (each actor had it's own work thread)... the server would freeze up randomly with only a handful of connections in a test scenario... changing to an event-loop using an async thread-pool resolved most of these issues (this was before node).
We've been running a Koa.js API server using Cluster in production for over a year now with no hiccups (on a Windows machine).
I've been thinking about making the switch to iisnode, as it handles clustering, graceful shutdown and zero-downtime from within IIS (and does a couple of other things). It uses named pipes to proxy connections and also supports web sockets among other things.
With the nodeProcessCommandLine configuration setting, you can pass parameters to node (e.g. --harmony), use babel-node or io.js.
See: http://www.hanselman.com/blog/InstallingAndRunningNodejsAppl...
A blog post I wrote a while ago: https://shelakel.co.za/hosting-almost-any-application-on-iis...
If this had compared Node.js with clustering and async vs a synchronous language like Ruby, it might have been interesting. Maybe. But non-asynchronous operations in Node are an antipattern that core node contribs are trying to remove (the -Sync in node stdlib are trying to be removed).
Good coding conventions, promises, generators, and async/await are your friends for making callback hell go away.
Yes, if I'm writing a web server I want everything to be async. But, just as I want to share JS on the server and the client I also want to use JS as my build language. At least for me, I find it much faster to build using a sync style.
Maybe I just haven't learned the async way but for example I tried to make a build system using node. I needed to spawn out to a builder to build some stuff, copy files, spawn git in various ways to see if repos are dirty or clean, git add, git commit, and a few other things. I found it a massive nightmare and after 2 days I switched to synchronous python for my building. Was done in 2 hours.
If there's some articles or tips that will make me as comfortable at async for building as I'm as sync in python then please point me at them. But, for whatever reason, I'm struggling with async for really complex tasks. (and yes, I'm using promises)
If you really prefer synchronous style, and don't mind the drawbacks, there are plenty of languages out there that'll work better for you. PHP, Ruby, and Python readily come to mind.
For example use a global object called State that contains information about the list of build tasks to be run and the state of each of the stages for the tasks.
I suppose though that you could always just call process.exit() when a callback was called or a promise resolved. Personally I'd love a file-level await statement.
At Nginx conf the Nginx developers where showing interest to bring Javascript to the platform. They said that they will take similar approach that OpenResty uses (aka no callback hell).
Instead of taking a callback, a function returns a promise, which can be daisy-chained to do work.
Ex:
file.read().then(console.log);
Or using the example in the article:
var a = fileA.read();
var b = fileB.read();
promise.all([a,b]).then((data) => console.log(data[0], data[1]));
let [a, b] = await Promise.all([asyncA(), asyncB()])Also Sync versions of calls in NodeJS are likely going to be deprecated, thus this won't even be possible in NodeJS going forward.
OS-level multitasking won't be able to achieve the same level of concurrency, but the simplicity and maintainability of the application will go up. The right choice depends on the needs of the application, of course.
Both evented and non-evented approaches have their place, and most server-side languages allow development with either approach: Ruby, Python, C, Java all have solid options for evented and non-evented solutions.
Do you have a source for this claim? I've seen it repeated many times, especially in the node.js community but I've yet to see any evidence to back it up. From what I've read, a synchronous threaded model can be just as fast as an event-based system [1].
These days you could actually reasonably have 10k threads. In theory switching between threads shouldn't be much different performance-wise than switching between callbacks in an event loop (either way you take some cache misses), and the thread stack is probably more cache-friendly than scattering objects all over the heap (and certainly easier to use).
But now you have the problem that synchronization between threads (whether mutex locking or by lock-free algorithms) is complicated and surprisingly slow, specifically because you have to worry about all the ways simultaneous memory access might confuse the CPU or its caches. Whereas with single-threaded async each callback is effectively a transaction, without requiring any slow synchronization.
Of course if you're doing single-threaded async then you probably aren't fully utilizing even one core. You see, even if you think you are doing everything in a non-blocking way, that's not really the case all the way down the stack. If you try to access memory that is paged out, guess what? You are now blocked on disk I/O. And because you aren't using threads, the OS can't schedule any other work while you wait. And even if you're pretty sure you never touch memory that is paged out, you surely do sometimes touch memory that is not in the CPU cache, which also takes a while. If your CPU supports hyper-threading, it could be executing another thread in the meantime... but you don't have any other threads.
And then multicore. The previous paragraph was a lot more interesting before multicore, but now it's just obvious that you can't utilize your CPU with a single thread.
The heavy-duty high-scalability servers out there (like nginx and I'd guess HA Proxy) actually use both threads and async, but while this gets the best of both words, it also gets the worst: complicated synchronization and callback hell.
Basically, all concurrency models suck.
This is touched upon in the slides you linked to. On slide 62 (SMTP server) a point says, "Server spends a lot of time waiting for the next command (like many milliseconds)." A malicious client could send bytes very slowly, using up a thread for a much longer period of time. If the client has an async architecture, it can open multiple slow connections with little overhead. The asymmetry in resource usage can be quite staggering.
Every second:
1. Open 40 connections to the server.
2. For all open connections, send one byte.
Repeat indefinitely.
Steady state would be reached at 4,800 open connections. At 1 byte of actual data per second per connection, data plus TCP overhead would use around 200KB/s of bandwidth. The server would have to run 4,800 threads to handle this load. Depending on memory usage per thread, this could exhaust the server's RAM.
There are ways to mitigate this simple example attack, but the only way to defend against more sophisticated variants is to break the one-thread-per-connection relationship.
I'm not saying that the threaded approach is better, but that almost everyone comes around with some theoretical statement but nobody seems to care to find hard evidence.
The slide deck that bysin linked above is pretty good:
http://www.mailinator.com/tymaPaulMultithreaded.pdf
This is by Paul Tyma, who at the time worked on Google's Java infrastructure team with Josh Bloch and other people who know what they're doing. Apparently he found threads to be faster in a number of benchmarks.
Ultimately which is actually faster will always depend on your use case. Unfortunately this means that general benchmarks aren't all that useful; you need to benchmark your system. And you aren't going to write your whole system both ways in order to find out which is faster. So probably you should just choose the style you're more comfortable with.
Async is kind of like libertarianism: It works pretty well in some cases, pretty poorly in others, but it has a contingent of fans who think they've discovered some magic solution to all problems and if you disagree then you must just not understand and you need to be educated.
(Note: The code I've been writing lately is heavily async, FWIW.)
Code looks like this:
var sync = require('./sync');
sync(function () {
try {
var result = someAsyncFunc1.callSync(this, 'param1', param2');
if (result.something === true) {
var result2 = someAsyncFunc2.callSync(this, 'param1');
} else {
var result2 = someAsyncFunc3.callSync(this, 'param1');
}
console.log(result2.message);
} catch (ex) {
// One of them returned an err param in their callback
}
});
https://www.discovermeteor.com/blog/wrapping-npm-packages/
It feels a bit like "get that sync code off my Javascript lawn!" but once you get used to it, it's pretty great in practice, and good for introducing newcomers.
The more annoying con is lack of shared memory. A single process can be much less complex when it doesn't have to worry about messaging systems and off process caching.
The only real competition is transactional memory but it hasn't become mainstream yet.
I will say this, though, regarding the need for shared, mutable state: You can communicate by sharing memory, but you can also share memory by communicating. This is I guess what you allude to with database-only state, but it doesn't need to be a database. It could just as easily be a memcached server or some other fase key-value store.
I'd argue the root cause is... callbacks.
Asynchronous programming can be done elegantly, in a synchronous style using "async/await", originally (?) in C# [1], likely to be added to the next version of JavaScript [2], also in Dart [2], Hack [4], and Python 3.5 [5]. It can also be emulated in languages with coroutines/generators [6][7][8] (which in turn can be implemented by a fairly simple transpiler [9][10])
This:
function foo(a, callback) {
bar(a, function(err, b) {
if (err) {
callback(err)
} else {
baz(b, function(err, c) {
if (err) {
callback(err)
} else {
// some more stuff
callback(null, d)
}
})
}
})
}
async function foo() {
var a = await bar(a)
var c = await baz(b)
// some more stuff
return d;
}
[1] https://msdn.microsoft.com/en-us/library/hh191443.aspx
[2] http://jakearchibald.com/2014/es7-async-functions/
[3] https://www.dartlang.org/articles/await-async/
[4] http://docs.hhvm.com/manual/en/hack.async.asyncawait.php
[5] https://lwn.net/Articles/643786/
[6] https://github.com/petkaantonov/bluebird/blob/master/API.md#...
[7] https://github.com/kriskowal/q/tree/v1/examples/async-genera...
Not to mention, that the article's example is only comparing a small piece of work... A single node instance can easily manage 10K simultaneous requests... launching a thread per request, and you're going to hit resource bottlenecks at the CPU pretty quickly, compared to several node instances approaching a million simultaneous connections on a single server. Node uses not only an even loop, but a shared thread pool for isolation of work without blowing out resource contention.
I've seen issues with even a few thousand threads in poorly written simulation servers... going to an event-loop against a threadpool always worked out better under true load.
In the meantime, for those of us stuck with contemporary JS, there are mitigating code styles, in particular "early return":
function foo(a, callback) {
bar(a, function(err, b) {
if (err) return callback(err)
baz(b, function(err, c) {
if (err) return callback(err)
// some more stuff
callback(null, d)
})
})
}
It is however much clearer than the "traditional" style.
They don't handle streaming data, but they're not supposed to; they're supposed to model the traditional JS function call model (single value return or single value exception) which they do well. For streaming data use some other paradigm (perhaps Observables from https://github.com/jhusain/asyncgenerator), but that's orthogonal to Promises, not a replacement for them.
The only other reasonable complaint I've heard is that they aren't cancellable, which is legit. There are proposals for that, though (https://github.com/petkaantonov/bluebird/blob/master/API.md#...).
At least with Promises functions actually return, as opposed to the continuation-passing-style of callbacks. (CPS is a fine intermediate representation for compilers, but it really shouldn't be written by hand; that's why we have compilers!)
For example, let's take the wonderful async.js library. There are a variety of different tools at my disposal on picking how to run tasks in parallel. My favorite one is async.auto. With this method, I can define the asynchronous tasks and their dependencies, then async.auto finds the best way to run the tasks in parallel while still ensuring the tasks end up having their dependencies met. This is more in tune to declarative programming's ideal of defining what results you want, rather than specifying how to accomplish getting those results.
Now with ES7's async/await, everything is still sequential. Better than blocking synchronous programming, but it could be better.
Here's an example of some code I had to write to query a somewhat nasty Web API:
async function apiCall(method, path, qs, body) { ... }
async function getPaginated(path, qs, extractionFn) {
let data = [];
let pagePromises = [
apiCall('GET', path, Object.assign({ }, qs, { page: 1 }))
];
let firstPage = await pagePromises[0];
for (let page = 2; page <= firstPage.pagination.page_count; ++page) {
pagePromises.push(
apiCall('GET', path, Object.assign({ }, qs, { page }))
);
}
let pages = await Promise.all(pagePromises);
for (let page of pages) {
let extracted = await extractionFn(page, qs);
data = data.concat(extracted);
}
return data;
}
async functions magically return promises. You don't have to just await them, you can choose to do whatever you want with the resulting promises. And promises, being actual values and not just functions called at the right time, are much easier to compose.
Anyway, if you like the async.auto functionality, it wouldn't be very hard to write a Promise.auto that would let you do (compare to the async.auto example at https://github.com/caolan/async#auto):
let results = await Promise.auto({
get_data: async function(){
console.log('in get_data');
// async code to get some data
return [ 'data', 'converted to array' ];
},
make_folder: async function(){
console.log('in make_folder');
// async code to create a directory to store a file in
// this is run at the same time as getting the data
return 'folder';
},
write_file: ['get_data', 'make_folder', async function(results){
console.log('in write_file', JSON.stringify(results));
// once there is some data and the directory exists,
// write the data to a file in the directory
return 'filename';
}],
email_link: ['write_file', async function({write_file}){
console.log('in email_link', JSON.stringify(results));
// once the file is written let's email a link to it...
// results.write_file contains the filename returned by write_file.
return {'file':write_file, 'email':'user@example.com'};
}]
});
console.log('results = ', results);
let [ data, folder ] = await Promise.all([get_data, make_folder]);
I've written code with async.js, and it's nice. I vastly prefer Promises. The only thing I still use from async.js is the queueing stuff, and I wrap that up with Promises.
async function Promise_auto(clauses) {
let keys = Object.keys(clauses);
let results = { };
let clausePromises = { };
function runClause(key) {
if (!clausePromises[key]) {
clausePromises[key] = (async () => {
let fn = clauses[key];
if (fn instanceof Array) {
let deps = fn.slice(0, -1);
fn = fn.slice(-1)[0];
await Promise.all(deps.map(runClause));
}
console.log('start', key);
results[key] = await fn(results);
console.log('end', key);
})();
}
return clausePromises[key];
}
await Promise.all(keys.map(runClause));
return results;
}
(async () => {
let results = await Promise_auto({
write_file: ['get_data', 'make_folder', async function(results){
console.log('in write_file', results);
return 'filename';
}],
email_link: ['write_file', async function(results){
console.log('in email_link', results);
return {'file':results.write_file, 'email':'user@example.com'};
}],
get_data: async function(){
console.log('in get_data');
return [ 'data', 'converted to array' ];
},
make_folder: async function(){
console.log('in make_folder');
return 'folder';
},
});
console.log('results = ', results);
})();
=>
start get_data
in get_data
start make_folder
in make_folder
end get_data
end make_folder
start write_file
in write_file { get_data: [ 'data', 'converted to array' ],
make_folder: 'folder' }
end write_file
start email_link
in email_link { get_data: [ 'data', 'converted to array' ],
make_folder: 'folder',
write_file: 'filename' }
end email_link
results = { get_data: [ 'data', 'converted to array' ],
make_folder: 'folder',
write_file: 'filename',
email_link: { file: 'filename', email: 'user@example.com' } }
And Node has the equivalent of WSGI and Rack: Connect/Express middleware.
I understand that because if the funny scoping rules it means that threading is actually surprisingly hard? surely you'd want more control over your threaded event loops?