Node.js fanout pubsub server for real-time apps | Hacker News Reader

Node.js fanout pubsub server for real-time apps

github.com

54 pointsjazzychad7 comments

I'm wondering if the author or other readers have looked at this node.js library:

Coglan has put together code for ruby and eventmachine too, if you prefer those tools over javascript and node.js.

Use faye in combination with redis-node-client and you've got an efficient means to create a high-volume, low latency work queue.

40+ mbps of http traffic and upwards of 20,000 redis requests per second can be serviced by a couple of node processes, using only a few hundred MBs of RAM and a plain-jane Linux VM with access to multiple cores. Tests are ongoing....

I mention that library specifically as it's a pubsub implementation for http/javascript too.

Where's the difference between using this pubsub implementation vs. using Redis one? I think that overall Redis one is way better (stability, performance).

I don't know. It may be better, I haven't looked at it yet. I wrote this implementation last December when redis didn't have pubsub yet. I thought it would be nice to share it as an example of the simplicity and power of node/javascript.

How can you state that the stability and performance are better? I've been running one instance of this node pubsub server for 5 months without restarting it. You may be right, but have you actually benchmarked it?

can anyone explain why the message format uses fancy control chars like

    <channel_name>!<message><newline_character>
    foo!hello, world

why not json?

    {cname: "foo", msg: "hello, world"}

i mean, parsing fancy format may be a bit faster and needs less library code; and, of course, overhead. but from personal experience i know that fancy control chars mostly lead to perlish charsoup as soon as new features are implemented.

    foo!hello, world##from_id:213/5|sendername:chad//validuntil{20100429105700}?debug=true&nlformat=CRLF

i guess, overhead; am i right?

I don't consider ! to be a fancy control character. It's just simple.

Outputing/requiring json adds extra complexity and restriction that isn't necessary. Although, nothing says you can't send the messages as json (which is, in fact, what I do in my applications!). Letting the message format be completely free-form means you can treat it however you like.

The idea was to make it incredibly simple, and abstract enough (meaning no extra/unneeded logic baked in) so that it could be used in the widest set of applications. The application you build with it may well use json as its message format, but I saw no need to require a certain format.

Here is an example of a message I'm using in the 4squrevision example:

    4sq_checkins!{"iid":17295,"tid":13057779551,"location":"Maths Journey","sqlink":"http://4sq.com/cGG46G","user":"greeenpea","lat":null,"tstamp":"Thu Apr 29 08:20:33 +0000 2010","lng":null}

Simple.

I'm curious about the "fanout" name which immediately made me think of a message broker like RabbitMQ on the backend somewhere. I guess "fanout" here means 1 client to fanout.js to N >> 1 other clients.

There are a number of similar projects popping up like this one as of late for Node.js. NodeRed, fanout, dealer, etc.

That's great. Node.js makes previously "complex" things a little easier to setup and play with.

Reply on news.ycombinator.com