Hacker News Reader

Comment by benatkin | Hacker News Reader

redis is pretty badass for talking to other processes without really talking to other processes, and it's the #10 most depended-upon library on npm right now. http://search.npmjs.org/

It won't work for every problem, of course.

dnode is a good way to easily talk to other node.js processes without the HTTP overhead. It can talk over HTTP too, with socket.io.

node-http-proxy is useful as a load balancer, and a load balancer can distribute work between cores.

Finally, most of the node.js people I've met, online and offline, are polyglots, and are happy to pick a good tool for a job. But right now node.js has great libraries for realtime apps, the ability to share code on the client and server in a simple way, and good UI DSLs like jade, less, and stylus.

View on HN

Huh? how does any of this keep me from having to write callback spaghetti? If I send a call using redis or dnode or whatever, I have to wait for it, so that means a callback.

I feel you about the polyglot and tend to agree, but I think some people are really trying to force awkward things into node, like people attempting to write big full-stack webapps using it.

I didn't mention any of those libraries in my reply to your original comment. To help alleviate callback noise, there's https://github.com/caolan/kanso , which Fenn mentions in his blog post's comments, and there are EventEmitters, and there is Function.bind() and Backbone.js-style async, where you have methods that describe what each callback does. (Backbone is usable in Node.js but perhaps not as practical on the server as it is on the client.)

Also callbacks to me are kind of like parenthesis in Lisp. They're annoying but they're for the greater good. :)

"deal with it" is a good response. personally, I'd rather write callbacks than deal with threads any day. And there are plenty of approaches to make dealing with callbacks easier (async.js is one of many).

Actually, "deal with it" is fucking lame, isn't a response at all, and is a good example of why I use the term "fundamentalism."

Is your problem with threads or shared mutable state? Web applications should be stateless and can be written as long request-response pipelines on-top of a pool of actor threads, with the only shared state existing at either ends of the pipeline, probably hidden by a framework anyways.

The "deal with it" is the same for threads - if you're using a system with threads you have to deal with them and it's generally not too fun. Node is what it is. It's callback based. I didn't mean the "deal with it" in a hostile way, more like that's the kind of system node is so deal with it. Is node great for every kind of problem? No. Definitely not. Actors are great too (and they've usually got threads in them, but they are well contained). But for certain kinds of apps node and the careful use of callbacks work out great. I guess I was trying to counter all of the hating on callbacks - they're not so awful when you get used to them.

Tamejs makes it a bit easier to deal with. Life doesn't have to suck.

the thinking that async code == callback spaghetti shit has to stop.

There are plenty of async idioms that make callbacks a breeze.

I'll definitely admit that many of the inventive techniques that node.js users have come up with make dealing with callbacks less absurd, but the problem is that it's just polishing shit. It's extra hoops to jump through with no benefit, outside of the C10K fapfest.

Would all the developers writing apps on node.js who are doing 10,000+ concurrent requests per process please stand up?

Only the minority gives a shit how many Hello World requests your stack can pump out. Some of us use node.js because it's a) got some decent libraries, b) makes realtime easy, c) uses the same code server+client-side reducing context-switches, and d) gets quite enjoyable once you know what you're doing.

You obviously haven't written a lot of async code so not sure why you're so against the idea.

So again, to you, what benefit does everything being built on this async, single-threaded, event-driven model give you? Sounds like not much. You could get all of those benefits + much cleaner code using threading or fibers or actors, but that does not make for C10K badassness, so here we are, with our callbacks.

Also, I'd say I've written enough code on top of node.js to be qualified to comment on this. Here's some of it that's open source:

https://github.com/rbranson/glob-trie.js https://github.com/rbranson/twerk https://github.com/rbranson/node-ffi

I get simple scalability, server-client code reuse, easy realtime, and, despite what you or others may think, I like Javascript as a language. Node.JS was also a perfect fit for the scraping framework I wrote: https://github.com/chriso/node.io

At one stage I had the same train of thought that you have (http://chris6f.com/synchronous-nodejs / https://github.com/chriso/synchronous/blob/master/lib/protot...) - it would be nice to have the option of fibers, but it's not going to happen.

IMO async code isn't as difficult or ugly as you make it out to be. Is async code as easy to write and follow as sync code? No. Is it worth the benefits I've mentioned? For me, yes.

> Is it worth the benefits I've mentioned? For me, yes.

This sounds like a tacit admittance that you're willing to deal with it because you don't think you have other options, but you do. There is at least one CPS compiler for Node (TameJS), and there are other languages that allow for the same result, but more straightforward implementations of concurrent code (Erlang, Ocaml/LWT, Haskell). I'm not saying you should use though, but we can do better and we should, even if it's just compiling back to JS in the end.

If I can run one app with 10,000+ concurrent requests on one server, then I can run a hundred apps with a hundred concurrent requests each on that same server. You can say that doesn't matter either, but the cost of hosting a web app just dropped 99%.

but the cost of hosting a web app just dropped 99%.

The cost of hosting a webapp tends to be a rounding error in contrast to the cost of developing the webapp.

And now the cost of hosting a hundred webapps is just a rounding error in contrast to the cost of developing one. Isn't it nice living in the future?

The additional benefit is that I can take the same program and handle 20,000+ concurrent users on two servers— which is when I suddenly become very glad that my hardware costs are significant compared to my dev costs.

And now the cost of hosting a hundred webapps is just a rounding error in contrast to the cost of developing one.

That's a weird way to look at it, unless you're in the webapp hosting business? For everyone else there is usually only one webapp that they care about.

I can take the same program and handle 20,000+ concurrent users on two servers

Sorry to break it, but that's not how it works. Unless you have one of those rare webapps that never need to touch a database.

Really? Show of hands, now, who here cares about one (and only one) webapp?

Anyway, what's good for the webapp hosting business is good for web developers, and what's good for web developers is good for the technical ecosystem in general (and then the world). Of course going from VPSes to EC2s was a significant improvement. But that isn't as good as it gets. EC2 rates were cheap already, but when Az started the free tier it represented a significantly lower barrier to entry. That's good for everyone.

And seriously, come on. This is a way of making programs run faster, and not a little faster, but a hundred times faster. It's the very definition of technological progress. It's absurd that we're here arguing about whether it matters or not.

This is a way of making programs run faster, and not a little faster, but a hundred times faster.

Sorry, but if anything then that statement is absurd.

Faster than what? And where's that "hundred times faster" figure coming from?

It seems there's a bit of a misconception about the bottlenecks and cost structure in real world web applications.

Rails (aka the slowest web framework known to man) is popular because it trades hardware for development velocity. Hardware is cheap, developer salaries are not.

It's cheap until it's not. At a certain point, you just can't process more requests at once in Rails. That's your limit. And it's not much— 100, maybe.

But node multiplies that, a lot. Which is nice, because you know it won't break or slow down if a bunch of people use it for some reason. And so you don't have to re-architect your system for a while longer, which is valuable time.

But node multiplies that, a lot.

Yes. Rails is measured in hundreds per second. Node in thousands per second.

The point that you still seem to be missing is that the monetary amounts involved have normally turned into a rounding error long before you reach a traffic-volume where this difference becomes relevant.

Or, in other words, hosting a "webapp" already is nearly free in terms of hardware.

I have an server monitoring application that does this, currently I've tested on a vm with 4000+ clients with 1 server (all on the same vm), without any errors (CPU load spikes, but memory usage was actually low, I believe ~200MB for the entire test).

4000+ requests being sent every second to the server in total, the server is a single node process.

Reply on news.ycombinator.com