Escape from Callback Hell
ianbishop.github.com
ianbishop.github.com
They're just continuations, seriously, what's everyone's problem? You define a function, it gets access to the current scope, it defines the rest of the program flow.
If you feel like your code is nesting too deep, you define the function elsewhere and just reference it by name. Then you don't get access to the current scope.
Why is this so difficult to people?
The big problem with callbacks is that they hold dynamic state and behaviour but, unlike other dynamic (and many static) objects in most languages, do not offer any interfaces to manipulate and reason about them. That's what higher level abstractions like promises provide.
Sure they do. You just need to start thinking of javascript as a functional language and all this stuff becomes much ... simpler.
First of all, why would you even need to reason about a function's internal state? That's a sign of a leaky abstraction.
Furthermore, every time you want to manipulate the internal state of a function, what you're really after is defining a better API to provide arguments to said function.
And if you still for some reason need to dick around with a function's internal state, just use partial function application.
If all you have to implement async operations are plain callbacks then yeah, they are not really an abstraction of anything, and you can probably call them 'leaky'. Which is why you need to create real abstractions around them, like promises.
I think we basically agree.
You never change code in real life? You don't always know what functions are going to be async up-front. They might become async later, 4 levels down, when someone decides to do some IO. What happens then? Exactly. You have to ripple that code all the way up to the top of the API.
And there isnt some magical point at which a change becomes large enough that it warrants a refactoring of unrelated code just because of the control flow.
Consider the simple case where one wants to look up a value synchronously if one has it in memory, and go get it from storage asynchronously if one doesn't. That's a natural thing to want, but it's problematic in Node. The problem is not syntactic—you can easily write a function that calls back immediately in the one case and asynchronously in the other. It's that sync and async semantics don't compose well, so when you start to do anything a little complicated (e.g. launch N requests and call back when all N have either returned or failed), the two trip over one another. Working in a Lisp that compiles to JS, I had to write some surprisingly complex macros in order to get correct behaviour. I wouldn't dream of writing that JS code by hand.
Simply slabbing on a promise inside such a function only works if the function is pure to start with. Usually that's not the case with slow functions in javascript since the slowest thing in javascript is messing with the DOM, i.e global state.
And don't say "monad transformer stacks" somehow solve this problem in an easier way.
You get the performance benefits of non-blocking code. The simplicity benefits of blocking code. And (almost) none of the threading hell you get in imperative languages.
This is some documentation of the scheduler: http://blog.ezyang.com/2013/01/the-ghc-scheduler/
I find the "oh you changed tiny piece of logic, you should totally have to refactor every line of code that follows it" very strange. It obviously sucks, there are far better ways to handle it and they are slowly making it into the language.
var a = await someBigFunc()
do
a <- someBigFunc()
a <- doFirstThing()
b, c <- doSecondThing(a)
doFinalThing(b + c)If every function is effectively impure, it still sucks.
However currently javascript has a purely event driven model and this means that you have trouble fitting in even a purely CPU bound computation, as it would block the responsiveness of your UI (or other concurrent requests if your are doing server side JS). That's why there was a need for creating something like WebWorkers.
"there! See how much cleaner that is?"
No, I don't, and this stories solution in particular, is particularly ugly.
I wish `let` was a little different, syntactically. With something closer to what OCaml does, you don't need to surround everything in parens and braces. This would take all the visual grossness out of it (which I think is the biggest problem people really have):
let (cb = function(x) {
...
}) {
foo(x,y,z,cb);
}
let cb = function(x) {
...
} in
foo(x,y,z,cb);
[Edit: formatting.]
If you start seeing do1() do2().. or cb1(), cb2() and so on function that is the "hell" everyone is talking about. Of course you should name your functions better -- but that's not the point. Logically you might not need an extra function but you are forced to add it because of the way your framework forced you to handle IO.
That they're badly made continuations, without support from the language.
>If you feel like your code is nesting too deep, you define the function elsewhere and just reference it by name. Then you don't get access to the current scope.
At the cost of moving stuff out of where it's invoked, so making code harder to read.
The problem with callback hell is that is pushes the programmer to write FOR the machine, in the way the machine likes it. Those things should be an implementation detail, and in good languages, are.
Once I started using the async library, all of these problems everyone is illustrating basically disappeared.
A real continuation captures the entire execution context including the current call stack.
Instead what you get in javascript is a stack that's completely meaningless. There's no way to automatically route your exceptions to the real calling context, and there's no automatic way for you to be sure you're seeing all of your callee's exceptions.
If you really want to be sure you'll hear about the ultimate success or failure of an asynchronous call, every single asynchronous step needs to manually install it's own exception handler, trap any exceptions, and pass them back via a failure callback. You're basically doing by hand what the runtime environment is actually supposed to do for you (that being the whole point of exceptions).
Let's say you have 3 ajax calls going in parallel. With $.when, you can attach callbacks to arbitrary groupings of those ajax calls (callback1 runs when ajax1 and ajax2 are done, but callback2 runs when ajax1 and ajax3 are done).
I first learned about promises in Trevor Burnham's excellent book Async Javascript (http://pragprog.com/book/tbajs/async-javascript) and it is still the best explanation of promises I've ever read. If you like this article and are interested in reading further about promises or the asynchronous nature of Javascript in general (both for browser and node.js), I highly recommend you check out this book.
I got to write some firefox only code recently and added a delay to a function by just adding
... some code
setTimeout(continueFun, 5000)
yield;
... more code
More complex things are also easy to do as well, such as implementing a node.next(event) method, so you can simply do var click = yield document.next("click"); instead of the whole event listener shebang. The feedback example (https://github.com/eligrey/async.js#asking-the-user-for-thei...) shows how much of a difference using yield can be versus callbacks or even promises.
Consider that with things like jQuery deferreds I can start a request in one function and pass it around adding handlers in others...something very hard to do with the old CB model. Maybe later on you have some optional action that you only want to occur if a previous action had completed. This sort of thing is easy with deferreds. It makes bundling together disparate actions easier as well as handling series of things as you mention.
Having an abstraction layer in there also has a lot of potential that is (somewhat) untapped. There is a lot of possibility for library authors to add excellent error handling or debugging capabilities in there - things that are currently a bit painful with POC (plain 'ol callbacks).
I agree generators have strong possibilities in this area- I would note thought that they are not an alternative to promises - the two are orthogonal, and in fact they work quite well together. Just take a look at what some other Mozilla folks are doing with http://taskjs.org/ for example.
https://github.com/fzzzy/pavel.js
The only caveat is that you do have to write "yield receive('pattern')" instead of just "receive('pattern')".
https://github.com/caolan/async
I've only used it with node.js, but it's supposed to work in web browsers as well.
It allows you to think a little more procedurally ("waterfall" is especially handy here) while writing CPS code. Very good.
Memory management, by comparison, is conceptually simpler because... well... the concept is simple. Allocating and de-allocating resources, while tricky at scale, is something for which everyone (including non-programmers) probably have existing mental models for.
Event driven programming, on the other hand, is a slippery high level concept. There are relatively few analogues for it in the "real world", and therefore requires additional mental gymnastics to internalize and understand.
Essentially, we need to 1) follow the execution pattern of event driven code (annoying), while at the same time 2) "visualizing" or conceptualizing a fairly un-natural manner of abstraction.
That doesn't make it any less difficult.
You really need to read the Deferred implementation if you are going to use it. Otherwise you are asking for trouble long term. Of course, the other issue is that you may run into challenges explaining deferred's to your co-workers. :)
Twisted explored some cool ideas where you basically would write asynchronous code in an interative style using a blend of iterators and generators. Sadly until Javascript has those capabilities in every browser (and not just Firefox) I don't think it is possible.
http://code.google.com/p/v8/issues/detail?id=2355
If v8 implements this, both Firefox and Chrome as well as node.js will have yield, enabling libraries like taskjs [1] to be used.
From taskjs.org:
spawn(function*() {
var data = yield $.ajax(url);
$('#result').html(data);
var status = $('#status').html('Download complete.');
yield status.fadeIn().promise();
yield sleep(2000);
status.fadeOut();
});
Just transmitting data back and forth may play well to the strengths of your abstraction. But we have other uses with Timers that should also need such abstractions.
With Timers I have other needs like delaying the execution, resetting the delay countdown, stopping it before it executes it at all (like cancelling it), and finally with an Animation class I needed a way to finish executing a string of events in an instant in order to start a new animation. Also the Animation had other Animation versions at play that could need to be sped up before a new Animation started.
In .NET they seem to have a handy feature that waits the code to run before proceeding that comes into play with their .NET version of the Reactive Framework.
As far as I can tell, it's tough to really solve it. JavaScript doesn't have extra features like .NET does. We are more limited in what we can do. In Dart they have a version of this called Future that has been streamlined recently. As simple as it may seem to be, it comes with other related abstractions called Streams that altogether make it a bit daunting to escape from that hell only to land on the fire outright.
Simon Marlow captured this well: http://stackoverflow.com/a/3858684/83805
"the abstractions that threads provide are essential for making server code easier to get right, and more robust"
https://github.com/Sage/streamlinejs http://en.wikipedia.org/wiki/Continuation-passing_style
There's a gotcha with the progress handler. If you try to call the progress handler before the progress handler actually gets a chance to attach itself outside the function, it'll never actually fire. Some of the bugs with using promises are rather subtle.
As for pure JavaScript, dealing with callbacks is definitely not fun.
Here is a simple example in some Node.js code I've just written (I've never written any before this little project).
https://github.com/Offler/node-bundler/blob/master/lib/bundl...
As you can see you just need to write in an OO manner and use function.bind(this) and Bob's your uncle. Really don't get the difficulty.
function getItem(id) {
return $.get("/api/item/" + id);
}
$.when(getItem(3), getItem(4)).then(handleSuccess, handleFailure);
Future prediction: Someone will create code similar to the one above but inside a for-loop calling getItem hundreds of times instead of just 2.
I'm not sure I'm convinced if someone doesn't know how to do a bulk operation they're more likely to look because they've got an extra callback or two.
function createItem(item, successCallback, errorCallback) {
var req = $.post("/api/item", item);
req.success(successCallback);
req.error(errorCallback);
}
Honestly, I didn't like the examples in the article very much. Not only do they fail to show the major advantages promises have, but he also uses named functions everywhere and that is quite silly.
Really what I meaning to say is that it's much clearer as to what is happening. I was trying to point out that success and error were actually functions on the deferred returned from $.post, not just functions you can pass into $.post (as they once were - hence "circa 2009").
The methods and approaches jQuery provides should not be used as a mainframe to achieve your goal, they should only be used as helpers here and there if ever.