Is node.js best for Comet? | Hacker News Reader

Is node.js best for Comet?

amix.dk

26 pointsolegp26 comments

"best" is a very subjective word.

If we're talking about performance, I'd venture to guess it goes something like this:

C > Java > Scala/Clojure> node.js > Python > Ruby

But then there's all of the other tradeoffs, like how error-prone each environment is or how long it will take to code and debug in each language.

Note that Urban Airship recently did some research and found that Java with direct NIO code worked the best for them, not Netty: http://blog.urbanairship.com/blog/2010/08/24/c500k-in-action...

I think the ultimate perfomance is found in pure java.nio/C/C++ solutions, but I think having a bit slower perfomance and higher abstraction is better since it makes it much easier to maintain and debug the system. My general impression of java.nio is that it's very low-level and a generally hard to code against and it's the main reason why we didn't choose it.

This said, I think node.js offers great usability while perfomance is pretty good. So if I was developing a new comet solution I would give node a go - you can always rewrite to something more low level once you begin to hit limits. IMO going after java.nio directly is a premature optimization and most projects won't hit limits with node.js.

It also heavily depends on what kind of state you're maintaining, whether any of it needs to be persisted or shared across connections, and how much CPU load the processing outside of pure connection handling will produce. Having a suitable scalability mechanism may be worth more than reduction of per-connection-event CPU cycles. (scaling beyond more than one core, scaling beyond more than one server)

Why is Ruby last? EventMachine should work pretty well for most purposes I think.

Eventmachine can juggle connections fairly well, but the common MRI problems kick in once you start actually doing something with these connections. Namely heap fragmentation followed by unbounded heap growth which results in stop&go operation during gc runs - until the host eventually swaps to death.

Other ruby impls (REE, JRuby or rubinius) might be better in those regards. But I wouldn't trust MRI with a long running process that might get under anything but the lightest pressure.

Erlang deserves a mention too, although ultimately I'm a bit pessimistic about their chances to break out of the small niche language space.

Me too. Awesome language/platform, but probably a bit too weird for where the industry at large is at these days, unfortunately.

Anybody else coming to ErlangCamp in Chicago this weekend, BTW?

I think it's beyond just being 'weird' - it's genuinely quirky for many things, and doesn't have good library support. IMO, this makes it sort of 'uneven' - really good at some things, and bad for others. The popular languages all seem to be those that are 'pretty good' for a lot of things, so once you learn the language, you can get many things done with it. Even if the language is perhaps not the best for any given task, it's good enough to at least crank out something that works ok.

The "doesn't have good library support" factor is mitigated by its "ports" - Erlang is happy to run arbitrary processes in C, Python, Lua, whatever in a bubble and just be the fault-tolerant network glue tying them together.

I seem to be fond of languages that make polarizing trade-offs but encourage multi-language programming: Lua, Prolog, Erlang, awk, etc. Use them when they're the best tool for the job, and when they aren't, they'll happily co-operate. Win win. (Requiring people to know multiple languages seems like it'd be a burden, but said languages can also be a lot smaller because they can avoid being responsible for what they're bad at. And, isn't that ultimately the point of DSLs?)

I thought you just liked languages that start arrays at 1...

If you have to glue stuff together in C or with pipes, it's going to be just as easy to do it in Tcl, Python, Ruby or whatever though, which also have more libraries to start with, and are fairly "promiscuous" in their own right.

I'm not sure I buy the 'glue language' thing: that's what both Tcl and PHP (and Perl to some degree) started out as, but it seems that people end up wanting to do everything in the higher level language that they can, because it's a lot less effort.

Something that might be really cool is if one of the efforts like Reia took off and made something a little bit nicer looking that sits on top of the Erlang run time.

> I thought you just liked languages that start arrays at 1...

Ha. If anything, I'm big on languages based around pattern matching. :)

Oh, and Erlang's ports are better than Unix pipes - it's more like a process run in isolation with asynchronous IO, and which the local Erlang runtime can restart as necessary. I changed the wording a bit to make that clearer.

I prefer working 100% in higher level languages and then moving hotspots down to C, etc., but it seems like languages designed to be glue languages make the transition easiest (because they have a kind of inherent modesty).

I never got into Tcl, but feel a common mindset in Ousterhout's essays (such as http://home.pacbell.net/ouster/scripting.html).

I think Erlang would be a great fit for comet, especially since it's already proven itself for Facebook chat. Erlang as a language is quite alien to me tho' and I generally prefer Java over it, but I am pretty sure that a Erlang solution would scale very well.

Yeah Erlang is great for a lot of things, including Comet.

I've been working on my own framework called Manos de Mono. It runs on top of Mono and is styled after node.js. I am hoping that it will be really good for these scenarios and the Mono runtime will address some of these concerns.

All that being said: node.js is impressively good. Its a beautiful piece of software.

http://github.com/sustainablecode/ringo-evented might be worth a look to get the best of both worlds

Socket.io + node.js makes it super easy. It abstracts away all of the transport protocols so you can just focus on writing your application.

Same goes for TornadoSocketIO though, so this isn't unique to node.js.

This would really be best handled as two separate questions -- "is a fully asynchronous approach best for Comet" and "is the node.js implementation best for Comet" are probably very different discussions. And again, "best" could mean either performance/scalability, usability, or some combination thereof.

Looks like he was already using Python - why didn't he try Tornado, which is built for purpose?

We tried to use Twisted and it did not perform that well (too much CPU usage, too much memory usage). I think a similar limitation will be hit with Tornado, but I haven't tried it out. Most of Plurk is implemented in Python tho' and I would love a great Python solution.

I work with Tornado daily, and while I don't have the traffic you guys do, I can say that Tornado's a lot more pleasant to program in than Twisted. asyncronous, for the end user, is implemented as a decorator you add to the URL handler's get() method. You then have an 'on_new_messges' method on the URL that's the callback to run when you've got something to push.

Best of all, there's about eight sample projects that are included in the source code. Ie, they're maintained, always work, and you always have a 'certified' example of how to do something. It's neat, and the community are friendly to boot.

Beyond websockets, I've found that SocketTornado.IO works great out of the box and supports multiple transports with ease.

http://github.com/SocketTornadIO/SocketTornad.IO

What about Scala and Lift?

socket.io with node.js is working out very well for me. If you want to throw redis and rails in the mix, check out github.com/maccman/juggernaut

Hello. It looks like you haven't participated much here. The HN community usually shuns these sorts of comments, because they don't really add to the conversation -- an upvote is already a "+1".

Reply on news.ycombinator.com