If you're doing Node.js, Caolan's async library is pretty much part of the standard toolkit.
I know Caolan's been thinking about this and reworking it for a while, so I'll be interested to see whether it manages to see significant takeup.
I know Caolan's been thinking about this and reworking it for a while, so I'll be interested to see whether it manages to see significant takeup.
- promises: about 20% of the room.
- async: about 80% of the room
For me async.waterfall([list of functions]) is little nicer than 10 chained .thens().
And people advocating promises still keep saying it keeps things flat. No it doesn't, we're already flat because we're all using async. Stop pretending async doesn't exist and isn't massively popular.
And way, way better documented. Q.spawn what? And this is the best promises library?
Stack Overflow question: Simplest fs.readFile example with generators and Q?
Current only answer:
Q.spawn(function* () {
…
var data = yield Q.ninvoke(fs, "readFile", somefile);
…
});
var data = _.wrapCallback(fs.readFile)('myfile');
- What does 'ninvoking' something do?
- Shouldn't I just be able to to put the variable declaration outside of the scope?
- Why do competing Open Source implementations of the same standard exist? Can't there just one reference implementation?
That's not the future.
I might be really ignorant here. I probably am - I could read a shit tonne of docs to work out what this strange beast does and technically someone can probably do a better job answering that Stack Overflow question. But nobody has, because very few people know how to operate the current state of the art generators/promises setup.
From the Q docs: "If you have a number of promise-producing functions that need to be run sequentially"
No, I don't have a number of promise producing functions. Nobody in nodeland has that. I just have functions. I could read about turning them into promise producing functions, and calculate whether this abstraction layer is adding value, but then again, I could do productive work with async.
And from the looks of it, Highland too.
async.waterfall([
fn1, fn2, fn3
], function(err, result) {
});
resultP = [ fn1, fn2, fn3 ].reduce(Q.when, void 0);
resultP
.then(function(result) {});
.catch(function(err) {});
The biggest value to me is being able to avoid the passing around of callbacks and them relying on varying conventions (some async are function(args, ..., callback(err, data)), some are function(args, ..., callback(data), errback(err)), some are function({success: callback, error: errback})).
Promises solve this problem by not passing around callbacks _at all_. Instead you return the promise and let the consumer attach the callback itself. And once we have promises widely available and part of the standard library, the calling conventions will be standardized too.
EDIT: I agree that as it stands, the lack of standardization of promises (jQuery's are mutable, for instance) is a pain, and that documentation could certainly be better.
var Promise = require('bluebird');
var fs = Promise.promisifyAll(require('fs'));
Promise.spawn(function* () {
var data = yield fs.readFileAsync(somefile);
});
My favorite example is doing a diff using an async diff service which provides a function `svc.diff(string1, string2)`. But also imagine that you need to preprocesses the files using the sync function `removeEmptyLines(buffer)`. This is how the function looks like when implemented using Bluebird:
function diffTwoFiles(f1, f2) {
var file1 = fs.readFileAsync(f1).then(removeEmptyLines),
file2 = fs.readFileAsync(f2).then(removeEmptyLines);
return Promise.join(file1, file2).spread(svc.diff);
}
- Isn't data a scope down?
- https://github.com/petkaantonov/bluebird/blob/master/API.md is API based, not task based. async is API based too, but the API has names like 'waterfall' and 'parallel'. I can click them because I know what they mean.
Promises just makes me feel like I'm reading about and endless series of abstractions.
I didn't quite understand the comment about data being a scope down. What do you mean?
Yes, promises do have quite a steep learning curve :/ However they're a lot more flexible than a utility grab-bag of functions that never quite fit the problem you're having. By that I mean I often had to massage my functions (by creating new closures or using bind etc) to make them fit the signature that async requires.
I wrote a bunch of examples for common tasks here - http://promise-nuggets.github.io/
For me, nearly everything .waterfall(), .each(), or .parallel(), and has been for two years now.
Promises are not about syntax sugar. They're about utilizing the whole power of the language and providing a parallel for most features found in synchronous code:
1. Functions have return values
When using node style callbacks, we're ignoring the fact that the language was designed with functions that have return values. Instead we use half-functions. Its no wonder those compose quite badly - the language wasn't designed for that kind of composition. The language was designed to work with functions that take input values and return an output value. Callback-based functions do only the first half. Thats why to get them to compose we resort to a bunch of hairy helpers and boilerplate code.
Callback-based functions that don't return anything are seriously crippled in power, and promises fix that, restoring much of the power.
2. Errors can bubble like exceptions
When using node style callbacks, we must explicitly handle all errors. On one hand, this is a good thing: we should deal with all errors. On the other hand, its quite tedious: most of the time we can't deal with the error at the exact place it appears but must pass it up one level in the call chain.
Promises do the error bubbling automatically. We can attach the appropriate error handler at the appropriate place to deal with the error.
This simple feature results with tons of useful patterns, one of which is the ability to manage resources with constructs like C#'s `using` keyword. - https://github.com/spion/promise-using
3. Values in variables can be accessed multiple times
When using node style callback and event emitters, we must make sure to "capture" the value exactly when it comes. If we don't do that, poof, its gone forever - we missed it.
In contrast, promises will keep the value for us. If we need to access that value later, we can simply attach another callback handler. An example where this may be useful is a database connection:
We initialize the connection and get a promise for that connection:
var pConn = db.connect(host, port);
function query(q, params) {
return pConn.then(function(conn) {
return conn.queryAsync(q, params);
});
}
Now try doing this with callbacks :)
Promises can be used as flow control, but more importantly, it's an object that encapsulates asynchronous mechanics.
I like to see how async can launch an asynchronous operation, then allow listeners to be attached later to capture the result. Now, you may say that if you want to attach listeners to capture result, you'll want to use event emitters. True, but event emitters has its own problem because event emitting and attach listeners are synchronous. What if the event was emitted before any body has a chance to attach listener to it?
Promises does not have these issues.
Where the line checks for `window.Promise`
var outcome = Boxon();
fs.readFile( "test.txt", outcome );
// Attach listener later:
outcome( function( err, content ){ ... } );
Boxon objects are light compared to promises, but they interop well. Best of both worlds!
In short, its API is nearly as extensive as Q's with essentially no overhead—bluebird is hardly more expensive than callbacks, while Q is something like 10x slower than using callbacks.
I ask because I wrote some Lisp macros (I work in a Lisp that compiles to JS) to implement a few async patterns I need, and making sure that exceptions are trapped and threaded into the callback chain correctly was the most complicated part.
From a practical perspective, it doesn't make sense to try to catch exceptions in asynchronous code, anyway. Once you do something asynchronous, you lose the stack and thus the try block. The way to catch thrown exceptions in asynchronous code is with domains, which something as low level as async would not be expected to handle.
Sure, but what if the error is thrown at you as an exception in the first place—which happens a fair amount, because that's how the JS runtime tells you when something is wrong? How do you get from there to the callback way?
What the Lisp macro I mentioned does is generate a separate try-catch around each block of code that runs at a different time and thus might throw an exception that would not otherwise get caught. In that way it catches every exception that's thrown, converts it to an error object, and passes the error back through the callback chain. The async library could do the same, albeit with a lot more code. I'm curious why it doesn't.
From a practical perspective, it doesn't make sense to try to catch exceptions in asynchronous code, anyway.
I don't think that's right. Asynchronous code is just synchronous code that runs at different times. Each block of synchronous code can generate exceptions. I agree that if you don't catch them then, they become useless; but you can catch them then. The reason this is not a "practical perspective" in JS is not that it doesn't make sense, it's that the language doesn't support it. Even the minimum code necessary to catch every exception involves so many try-catch blocks as to obscure the rest of the program. So no one writes such code by hand in JS.
Yet it is, I think, code that one wants, because without it you don't have a consistent error model. You end up having one model for first-class errors—the ones you detect and pass to callbacks before an exception has a chance to arise—and a second one for the dregs—the ones that come from any code that didn't know about or follow the callback convention (which, critically, includes the language runtime). The latter kind of error either crashes the server or gets caught by a top-level handler so it "only" crashes the request it was processing. That's a half-baked system.
Its on you to catch that, not your libraries. This shouldn't be terribly common, though. The only thing I can remember having to wrap in a try/catch in the codebase I work on is JSON.parse.
> The async library could do the same, albeit with a lot more code. I'm curious why it doesn't.
It couldn't, without domains. try-catch wouldn't do it. Domains are something that is not very well understood, in my experience, and expected to happen at a higher level than libraries like async.
As for domains, I don't know what you mean by them, but if they're catching errors at a higher level than the async library, my guess is that they must be some more sophisticated sort of top-level handler; perhaps something that keeps track of which async calls are in progress and attempts to bind exceptions back to their context? Whatever it is, it sounds complicated.
But what I understand least of all is how you guys all seem to write Javascript code that generates almost no exceptions. To me that sounds almost like bug-free code. No null references, for example? I get stuff like that all the time.
We write in CoffeeScript, where a null reference check is so astonishingly easy to write that you use them everywhere you might get a null. I'm not sure what other exceptions you're seeing. We do basically no math, so /0 errors aren't a problem.
> For example, I don't know why you say that the async library couldn't try-catch every place that an exception might occur
Let's build a typical function you might pass to async:
function(next) { request.get(url, function(err, data) { JSON.parse(data); } }
Let's assume the server doesn't serve JSON like we expect - so JSON.parse throws an exception. The only thing async could have wrapped in a try/catch is the main function, but we've fired off a request and then the call stack wrapped up, including the try/catch. Next, an event occurs that calls our callbacks, not going through async at all. That's where the exception occurs. The stack trace generated by that exception doesn't contain any code in the async lib, so it can't possibly have a try/catch active.
Domains are a way of fixing this. You create a Domain and bind callbacks to it - if that callback throws an exception, the Domain instead emits an error event.
Re null reference checks, to get behavior analogous to a null exception you have not only to check for null, but also pass back an explicit error if you find it. That's a lot more work than adding in an extra question mark. Null checks that do nothing but not crash are a mixed blessing; 90+% of the time they do what you want, but when they don't, you get a silent failure and a debugging goose chase. I'd be surprised if you told me that that never happens.
I took a look at Node.js domains and they do seem really complicated. If I were working in Javascript instead of having control over the language, I doubt I would use them; I would probably just crash-and-restart as one of the other commenters described. That's not a good solution, but probably the best tradeoff given the alternatives.
That said, we get very thorough testing from our large user base, and we quickly fix crashers. Our server proc crash rate is almost 0, brought up by occasional spikes on releases.
Boxon.cast( fs.readfile, 'myfile' )
.then( function( content ){ console.log( content ) })
.catch( function( err ){ console.log( "Error", err ) });