Kal – a clean JavaScript alternative without callbacks
rzimmerman.github.io
rzimmerman.github.io
For this Kal code...
task getUserFriends(userName)
wait for user from db.users.findOne {name:userName}
wait for friends from db.friends.find {userId:user.id}
if user.type is 'power user'
for parallel friend in friends
wait for friendsOfFriend from db.friends.find friend
for newFriend in friendsOfFriend
friends.push newFriend unless newFriend in friends
return friends
getUserFriends userName = do
user <- findUser userName
friends <- findFriends user
if (User.type user) == "power user"
then friends ++ parMap rdeepseq $ (getSecondNodes friends) friends
else friends
getSecondNodes firstNodes friend = do
secondNodes <- findFriends friend
diff firstNodes secondNodes
A lot of people prefer symbols over keywords, and I think that's mainly a readability issue. I personally prefer more keywords with good syntax highlighting, so that's what I went with (and why I almost immediately made a .tmbundle).
Thanks for your hard work!
eg.
getUserFriends = (userName) ->
user <- findUser userName
friends <- findFriends user
...
[1] - http://msdn.microsoft.com/en-us/library/vstudio/system.threa...
[2] - http://msdn.microsoft.com/en-us/library/vstudio/hh191443.asp... and http://msdn.microsoft.com/en-us/library/vstudio/hh156528.asp...
wait for friends from db.friends.find()
//vs
friends = await db.friends.find()
Is this a fork of the CoffeeScript compiler? Will it benefit from upstream changes/features?
If you get a chance (and use TextMate or Sublime Text), please give syntax highlighting a try (https://github.com/rzimmerman/kal.tmbundle)!
I think syntax highlighting helps a lot in this case, so try out the tmbundle in Sublime or TextMate (https://github.com/rzimmerman/kal.tmbundle) if you can.
For some time now, I've been accumulating async patterns I've needed when writing JS code in my monadic IO.js library[1]. The aim of IO.js is to provide higher levels of thinking about sequences of actions without worrying about whether they are async, and with a rich set of error management tools (for ex, exceptions are turned into recoverable conditions in IO.js).
The crux of the "callback hell" problem is, contrary to what many have claimed, is not the nesting that results when the callbacks need to be called in temporal order. That much is straightforward to deal with. The "hell" rears its head when you need to coordinate multiple such sequences that are running concurrently. The composable actions you get from a monadic treatment combined with CSP-style channels, are an expressive framework to build abstractions on (tldr - Haskell's implementation is awesome!).
For illustration, the framework in IO.js is flexible enough to implement a node.js web server that can express PG's "Arc challenge" concisely (though that challenge is practically obsolete). Take a look at [2].
For a simpler example, `IO.trace` is a straight forward way to generate a trace dump to console of a sequence of asynchronous actions. You don't need to "enable trace" for an entire app. You can choose to trace only a particular sequence. This is pretty neat when debugging. Stack traces are useless when dealing with async processes anyway.
[1]: https://github.com/srikumarks/IO.js [2]: https://github.com/srikumarks/IO.js/blob/master/examples/arc...
This situation is why I wrote DelayedOp: https://github.com/osuushi/DelayedOp . Of all the little reusable code bits I've written, this one has ended up in more of my projects than any other.
Basically, a DelayedOp is a lightweight manager that runs a callback after its "wait" and "ok" calls have balanced. It ends up being equivalent to _.after(), but it's more natural to use than manually counting your concurrent operations, and it has some conveniences for debugging, so that if a callback never fires, you can find out why.
The syntax is a little verbose and might be shortened to something like C#: `user = await db.users.findOne {name:userName}`. Would be a lot more clear to me.
Because, of course, there's no way to technically do that in anything that is JavaScript at its core -- at least as far as I'm aware of. Correct me if I'm wrong?
It says "This includes error handling via callbacks." -- not error handling via exceptions. Elsewhere it says "Any errors reported by fs.readFile (returned via callback) will be thrown automatically.", so maybe it converts certain callback error functions to exceptions? But in any case, that's not the same as throwing an exception inside of a callback, and having it "bubble up". Unless I'm misunderstanding (which would be great!).
In plain english, make a function that takes two functions makeADoThingFunctionThatHandlesErrors(doThing, handleError)
and returns a new version of the function that is internally wrapped in a try catch block with the handleError function getting called from the "catch" block. This is relatively straightforward to accomplish using closures/high order functions. This is one utility function you write once.
Finally you can do this with a whole "monadic" api such as jquery- create a function that loops through all the methods on an object prototype and wraps them in an error handler, returns a new version of the prototype you can inherit from.
Or instead of doing all that, you just use some library with promises (such as Q, or jQuery) that implements all the above, and all you need to do is write a single error handler function, and any error that gets thrown by any function in a promise chain gets caught and sent to that function.
wait for data from fs.readfile 'test.txt'
print data.toString
try
wait for data from fs.readfile 'test.txt'
print data.toString()
if data.length > 0
wait for data2 from fs.readfile 'test2.txt'
print data2.toString()
else
print 'too short'
catch e
print 'there was an error'
wait for resp from db.save()
var error = null;
var data, data2, resp;
fs.readFile('test.txt', function (err, data) {
if (err) return handleErr(err);
console.log(data.toString())
if (data.length > 0) {
return fs.readFile('test2.txt', function (err, data2) {
if (err) return handleErr(err);
console.log(data2.toString());
return closeout();
});
} else {
console.log('too short');
return closeout();
}
});
function handleErr(err) {
console.log('there was an error');
return closeout();
}
function closeout() {
return db.save(function (err, r) {
if (err) throw err;
resp = r;
});
}kal -o output.js -f beautify input.kal
to see the sausages being made. The http server demo is a good example (https://github.com/rzimmerman/kal/blob/master/examples/async...)
For this Kal code:
task getUserFriends (userName)
wait for user from db.users.findOne {name:userName}
wait for friends from db.friends.find {userId:user.id}
return friends
task getUserFriends (userName)
user ..= db.users.findOne {name:userName}
friends ..= db.friends.find {userId:user.id}
return friendsI chose it because it in Hebrew it roughly means something like easy/simple/BASIC.
edit: More importantly, does that make you more or less likely to use it?
Also, note that kal is mere `faeces` in Russian, not `shit`. I don't remember when I've last used the word outside of a hospital... or inside, for that matter.
I was once told it also sounds like something bad in Russian, so it can be a bit inconvenient introducing anyone with such name in Russian if not aware of this issue.
function getUserFriends(userName) {
return db.users.findOne({ name: userName }).then(function (user) {
return db.friends.find({ userId: user.id });
});
}
getUserFriends = (userName) ->
db.users.findOne(name: userName).then (user) ->
db.friends.find userId: user.id
getUserFriends = (userName) ->
db.users.findOne(name: userName)
.then (user) ->
db.friends.find userId: user.id
.then(function (friends) ->
somethingAsync(friends)
If that's not good enough you can implement higher level control flow abstractions much more nicely than with raw callbacks, e.x. https://github.com/tlrobinson/q-step
getUserFriends = (userName) ->
QStep(
->
db.users.findOne name: userName
(user) ->
db.friends.find userId: user.id
(friends) ->
somethingAsync friends
) function getUserFriends(userName) {
return db.users.findOne({ name: userName })
.then(function (user) {
return db.friends.find({ userId: user.id });
})
.then(function (friends) {
return somethingAsync(friends);
});
}I think a useful case would be the trickier stuff where Kal really shines. For example, doing async stuff inside of loops, conditional calls (where some paths need async waits and others don't) and error handling. I also think the Kal syntax might appeal to people who are less experienced with JavaScript and it's more eclectic features.
Q is a huge, monster library. It's 2x the size of caolan/async, which is already bloated. I personally think it's better to use https://github.com/mbostock/queue or another small, understandable library.
I grant you that mbostock's queue is even smaller, but all are so tiny as to be a trivial download even on mobile.
The important issue here is IMHO not the size of the libraries, but how much they help you write maintainable code.
http://blog.alexmaccaw.com/how-yield-will-transform-node
Definitely not as clean as a whole new language, but you can program like this today, at least in bleeding edge node and some recent browsers.
http://onilabs.com/stratifiedjs (disclosure: I work at Oni Labs)
It does extend JS with some additional syntax as well (new concurrency constructs as well as some syntactic sugar), but it doesn't alter existing JS syntax.
If I were you I would make a slight course correction follow in the footsteps of Python and Go and have one and only one correct way to do something.
When you say use two spaces, but you can use more if you want, make it an error to use more.
Pick the correct way to call a function. Pick the correct way to declare a function.
Keep up the good work.
Well done on making a whole language though. It's pretty challenging and I hope you do well.
Aside: This seems very similar to narrativeJS in concept http://www.neilmix.com/narrativejs/doc/
*jQuery or promises like API object, whereupon you place methods- some of which may be asynchronous. an asynchronous operation returns an object with a method that performs the next action and optionally takes a callback, passing in the results of the previous asynchronous operation as a value- thus flattening the callbacks out into a sequence instead of a nesting.
Constructed properly, you can create monad combinators/transformers, to do things like automatically wrap every step in your chain with exception handling. Using common JS promises libraries gives you error handling for free.
as for defining "join", help me understand. How is jquery "add" not essentially that? And why pick the fmap,join formulation instead of the bind,return formulation of a monad?
The reason I picked join is because (in my view, anyway) it's the easiest demonstration of how jQuery is not a monad. There's no way to even get something of type M (M a) in jQuery, because the jQuery lifting operation is idempotent. The same exact problem exists with return, but I've found that people have a hard time seeing it with that example.
The problem I have with you referring to jQuery as a monad, even by analogy, is that jQuery really really isn't a monad. It's only very loosely analogous, and that analogy is more likely to teach people the wrong things about monads rather than the right ones.
For the case in hand though, you can get to almost but not quite really technically a monad, and still cure most of the nested callback headaches. One way I think about monads is they are a strategy for turning nested functions into sequentially composed functions. Which is, pretty much what we want, right?
More to the point, I don't understand why we (HN and the JS community at large) keep having this discussion. Coroutines have been around since the 60s. This is a solved problem.
https://github.com/bjouhier/galaxy
Galaxy does not claim to get rid of callbacks, though... only to help make writing JS/es6 with async/await semantics easier. I've yet to try it out, but the instructions seem clear.
Javascript needs proper co-routines. Nothing short of co-routines will solve the issue. Please lookup microthreads, greenlets, fibres, etc.
Check how Q.nfcall works.
Let's assume a simple call stack, A -> B -> C. Now C would be a generator, so B has to loop over C instead of just calling C.
But what about A? Well if B is now looping, and if yielding out of a loop is how you cooperatively multitask, then A now has to loop over B. So you have to convert the entire call stack from A -> B -> C to A(for ... in B(for ... in C)).
You can argue that a compiler can make this invisible, however there are some serious drawbacks with that in that:
1) One solution would be to convert any call into a loop. How do you know which ones are actual calls (stuff like Math.pow etc?) 2) You only convert things that have generators in their call stack to loops, but how do you know that?
Both the above scenarios would depend on being able to infer the "generatoricity" of the called symbol. Something which is very, very hard to do for dynamic languages like Javascript.
Co-routines work differently. Co-routines save the entire call stack and restore another one on switching to somewhere different. Hence if you have A -> B -> C and C is some I/O that would have to go to some scheduler, then the A -> B -> C call stack is saved, the co-routine would switch to say, Main -> Scheduler, the scheduler would do its work, and when it's ready, it switches back to A -> B -> C where that co-routine left of.
But let me give you an example, again with Q.nfcall.
MultiplyAndSquare = (a, b, cb) ->
Multiply a, b, (err, product) ->
Square product, (err, square) ->
cb null, square #ignore errors for now.
nonCallbackVersion = (a, b) -> Q.nfcall(MultiplyAndSquare, a, b)
result = yield nonCallbackVersion(10, 20)Let's say you write some code A, which uses some other code B, which uses some other code C. C is also used by various other pieces of code like X, Y, Z, D, E, F, G, H, J, K etc. Now at some point you decide, well, you don't like callbacks in C, so you'll convert it to something asynchronous. You'll now just have to refactor A, B, D, E, F, G, H, J, K, X, Y and Z to make all their calls to C a generator iteration, and you'll have to convert any code that depends on A, B, D, E, F, G, H, J, K, X, Y or Z to be aware of that, any any code which uses those, to be aware of that.
MultiplyAndSquare = (a, b, cb) ->
Multiply a, b, (err, product) ->
Square product, (err, square) ->
cb(null, square) #ignore errors for now.
C = (a, b, cb) ->
MultiplyAndSquare(a, b, cb)
ABDEXY = (a, b, cb) ->
#Calls C
C(a, b, cb)
callersOfABDEXY = (a, b) ->
#Calls ABDEXY
ABDEXY a, b, (err, result) ->
Console.WriteLine(result)
#Refactored C. Returns a promise instead.
refactoredC = (a, b) ->
(Q.async ->
result = yield Q.nfcall(MultiplyAndSquare, a, b)
return result)()
#Refactored ABDEXY.
refactoredABDEXY = (a, b, cb) ->
#Change needed here, since C now returns a promise
refactoredC(a, b).then (result) ->
cb(null, result)
#No need to refactor callersOfABDEXYIt's not turtles all the way down, though. If you look way back, the 0.1ish releases of the compiler were written in CoffeeScript. I ported the compiler source over from CoffeeScript file by file as it matured and eventually got rid of the dependency entirely. Technically, you could start with the last release that was written in CoffeeScript and compile up to the current release.
Essentially self-compiling compilers can "remember" things that are nowhere to be found in the source code.
It seems like a number of folks in this thread have expressed interest in your process -- both of designing the feature set, and of gradually bootstrapping it away from CoffeeScript to become self-hosting. I'd love to hear more, if not here, then in a blog post, perhaps...
You have some examples with bloated err-returns, but where is that in your kal-code ?
replaces all the "if (e) return cb(e)" lines with cbe when you call the async function.
A whole abstraction over js just because of this, running away from the problem...instead of solving it on the spot, keep running.
Apologies if you saw these already. I'll add something like what you suggested, though.
On the other hand, that's so rude! Not every project has to be the next CoffeeScript to be interesting and worthy of appreciation. If you don't like it, just click on a different story instead of cutting down someone who chose to make something cool and share it with us.