Hacker News Reader

Comment by aston | Hacker News Reader

I've been waiting for this sort of thing for the longest. The first time I saw the callback passed to a callback of a callback's callback style in Node.js, I wondered why the code couldn't look as nice as the async C++ I'd written at OkCupid. From my lips to Max & Chris's minds... (telepathy!)

How long 'till the port to CoffeeScript syntax?

View on HN

It's funny, when Max started programming Tame.Js a couple weeks ago, I exchanged emails with a developer who tried to make something similar for CoffeeScript who couldn't convince others it was needed. He said the common response was that if you need something like Tame for async you're not "doing it right." (Obviously we disagree.)

I'd be more than happy to explore the addition of Tame.js-style CPS to the CoffeeScript compiler -- but there's a lot of prior work there already:

https://github.com/jashkenas/coffee-script/issues/241

https://github.com/jashkenas/coffee-script/issues/287

https://github.com/jashkenas/coffee-script/issues/350

Edit:

Things look a little less promising after running a simple test. This input JavaScript:

    while (i--) {
      twait {
        fs.readFile("one");
        fs.readFile("two");
      }
    }

Gets compiled into this resulting "tamed" JavaScript:

    var tame = require('tamejs').runtime;
    var __tame_fn_0 = function (__tame_k) {
        var __tame_k_implicit =  {};
        var __tame_fn_1 = function (__tame_k) {
            if (i --) {
                var __tame_fn_2 = function (__tame_k) {
                    var __tame_ev = new tame.Event (__tame_k);
                    var __tame_fn_3 = function (__tame_k) {
                        fs .readFile ( "one" ) ;
                        fs .readFile ( "two" ) ;
                        tame.callChain([__tame_k]);
                    };
                    __tame_fn_3(tame.end);
                    __tame_ev.trigger();
                };
                tame.callChain([__tame_fn_2, __tame_fn_1, __tame_k]);
            } else {
                tame.callChain([__tame_k]);
            }
        };
        __tame_k_implicit.k_break = __tame_k;
        __tame_k_implicit.k_continue = function() { __tame_fn_1(__tame_k); };
        tame.callChain([__tame_fn_1, __tame_k]);
    };
    __tame_fn_0 (tame.end);

... not so nice to work with or debug. The general conclusion of that series of tickets was that the code generation required to make this CPS transformation work with all edge cases is a bit too hairy to be worth it on balance. Depending on how much sequential async you're doing, YMMV.

Progress was made but looks like it's stalled atm. Since Tame has an endpoint maybe easier to just suck it in?

Edit:

That compiled script looks a wee scary. I need to be able to fully dive into a debugger with clarity and can't imagine if there were tens or hundreds of lines of this.

As jjm says, with Tame as a concrete, proven solution, it'd sure be great to have CS adopt something like it. Turning the problem "inside out" like this may be the best approach.

I.e., it has well-defined semantics and "only" involves JS rewriting, which is CS's forte.

The branch which attempted the defer keyword also had working code. The input was clean and the edge cases of what to do with return, try/catch, etc had been thought through.

The stopping issue was twofold: Coffeescript cares about the readability of both the compiler input and output and… Coffeescript cares about not inserting dynamic lookups and calls into the program which can add undefined performance impact.

Perhaps a fork of coffeescript that is targeted for writers of async NodeJS code would be able to safely trade those off. I think jashkenas believes that Javascript will benefit from many little languages springing up to solve specific problems.

It's on the list of todos to preserve the input line-numbering in the output code. This would mean some changes to the code emitter, and also to the code emitted to preserve the ordering of statements.

In tame/C++, debugging either compiler or runtime errors is straight-ahead, as the developer doesn't need to examine the mangled code. Now if only JavaScript had the equivalent of cpp's #line directive....

We're getting there ... maybe this summer.

https://bugs.webkit.org/show_bug.cgi?id=63940

https://bugzilla.mozilla.org/show_bug.cgi?id=618650

I think adding Tame style CPS to CoffeeScript would be amazing; it looks like an incredibly clean way to write the async code necessary for complex Node apps.

From my understanding of the prior work the issue with adding defer or <- to CS was that it required too much overhead to get right in all cases. Does TameJS' approach improve that overhead in any way, or is this essentially the same work that's already been explored for CS, broken out into a dedicated compiler?

adding something like this at the grammar level is a massive hack... we can just use coros

Not surprised. A rough transliteration of `huntMen` for example into CoffeeScript using normal node callback syntax is under 20 lines and relatively easy to understand:

  huntMen =(buffy)->
    soulmates =(buffy, cb, mates=[])->
      getMatches buffy, 10, (userids)->
        for u in userids
         do (u)->
          getThumbnail u, (thumb)->
            isPicAVampire thumb, (is_vamp)->
              unless is_vamp
                getPersonality u, (personality)->
                  getLastTalked u, match, (last_talked)->
                    soulmates.push userid: u, thumb, personality, last_talked
                    if soulmates.length >= 10
                      cb(mates)
                    else soulmates(buffy, cb, mates)
              else soulmates(buffy, cb, mates)
                      
    soulmates buffy, (soulmates)->
      #Do whatever you need to with soulmates

Obviously it's still more of a hassle to deal with than a more featurey async library, but there isn't quite the panic of trying to do the same thing in JS.

Edit: For comparison, the Tame.JS style transliterates to ~15 lines of CS— but presumably that would compile to dozens more lines of JavaScript.

In exchange, it is lying to you about what the program is actually doing, as opposed to the callback syntax which obscures nothing. So yeah, not surprised if you don't get too much traction with examples like that.

I believe this code is painfully unreadable - and that's despite the fact that CoffeeScript is very elegant and easy to read. (It's not you, it's async.) Further, maybe i'm misreading this, but it appears getPersonality and getLastTalked are fired in serial. Can someone who knows CS well fix that and reply? Thanks!

Ah, that's a good catch. Missed that. Calling those in parallel is legitimately a hassle to roll yourself, something like:

              ...
              isPicAVampire thumb, (is_vamp)->
                unless is_vamp
                  mate = userid: u, thumb: thumb
                  finish =-> 
                    return unless mate.personality? and mate.last_talked?
                    soulmates.push mate
                    if soulmates.length >= 10
                      cb(mates)
                    else soulmates(buffy, cb, mates)
                  
                  getPersonality u, (p)->finish mate.personality = p
                  getLastTalked u, match, (lt)->finish mate.last_talked = lt

Anyway, I'll grant you that code is pretty harsh. It's much nicer with syntax highlighting and wide tabs, but still, fair cop. The flip side is that that code compiles into JavaScript that does exactly what it says.

In exchange, it is lying to you about what the program is actually doing, as opposed to the callback syntax which obscures nothing.

I'm really fascinated with this thought that adding sugar/coroutines is "lying". Everything is based on abstractions; this is just another one. This is not a different "lie" than any other abstraction (assembly, C, OS, JavaScript).

That's a fair point— it's certainly subjective. I'd disagree that an abstraction is an abstraction, though; some are more deceptive than others.

Node comes with single-threaded, callback-based async that works quite well, but it's a bit of a hassle to use for complex stuff.

Nothing wrong with that— abstraction time! The async module, for example, comes with a few different callback-based flow-control patterns. The Buffy example would look something like this:

    huntMen =(buffy)->
        soulmates = []
    
        getSoulmate =(callback)->
            mate = {}
            async.series [
                (cb)->getThumbnail u, (thumb)->cb null, mate.thumb = thumb
                (cb)->isPicAVampire thumb, (is_vamp)->cb('vampire' if is_vamp)
                (loaded)->async.parallel [
                        (cb)->getPersonality u, (p)->cb null, mate.personality = p
                        (cb)->getLastTalked u, match, (l)->cb null, mate.last_talked = l
                        loaded
                    ]
                ], (err)->
                    soulmates.push mate unless err
                    callback()
        async.whilst (->soulmates.length < 10), getSoulmate, ->
             #Do whatever with soulmates

(I don't actually use async much, so forgive me if there's an error there.)

Async's abstractions are what I would call "honest". It's still using callbacks, it's obvious what the relationship between them is. The meanings of 'series' and 'parallel' are clear; I could write them out myself, it'd just take longer. Nothing about what this code does is being obscured by the abstraction, just made prettier. I know (or can easily work out) exactly what will be executed.

Tame is abstracting the same single-threaded, callback-based async, but it's trying to make it look like it's using threads. I know in theory it must be turning my code into callbacks which are being passed around, but it's deliberately trying to make that unclear. The result is that I have a somewhat worse understanding of what my program actually does.

To be clear, I don't have a huge beef; like you say, abstractions are necessary, and Tame seems fine to me. It's just my personal preference for more honest abstractions over more dishonest ones (no doubt heavily influenced by the fact that I actually love callback-based async, which I think puts me in a small minority.)

Reply on news.ycombinator.com