Fast polling using C, memached, nginx and libevent | Hacker News Reader

Fast polling using C, memached, nginx and libevent

amix.dk

71 pointsamix18 comments

>> "From what I have seen, all comet solutions are very hacky, quirky and very hard to scale. The scaling is especially hard since the whole web-platform is built around the request-response model."

I disagree about this. There's nothing hacky/quirky about comet, and it can scale really well. Some numbers from a Mibbit backend:

  Peak connected users: 4,000
  Peak HTTP Req/s: 1,500

  HTTP requests per day: 44 million
  Packets sent per day: 42 million
  Packets recv per day: 51 million

  (On a single 1.4GB VPS). Using *OMG* java!

If you're not using comet, that HTTP req figure would be way higher, as a far larger number of HTTP requests would be unuseful (no data available). Wasting time, bandwidth, etc As well as obviously adding latency. With comet, the data is available in most instances as soon as the server has it ready. With standard polling, the user has to wait for the next poll.

Comet is pretty easy to scale once you understand exactly what is going on, and how best to organize things. Mibbit uses a keep-alive long polling XHR setup which works really well. The keep-alive ensures that most of the time, you just have a tcp/ip connection open from server to client, with some HTTP request/response spam every so often, and the actual packets.

Having said that, I wouldn't use any of the open source comet things I've seen out there.

(I do plan to open source Mibbit at some point, which might be useful for people).

Out of curiosity, what transport are you using? Long polling or long-lived iframes? I find that the former doesn't give me low enough latency for my purposes and the latter tends to get quirky for me when I have a high spiked data throughput.

long polling XHR. Latency is extremely good if the server software is up to the task. As soon as the server sends an HTTP header and data response, it's there on the XHR response. (Also iframes mean memory leak, and possibly 'loading' spinner. Neither of which you get with XHR).

You can also do some neat optimizations like decide that a latency of 100ms is ok enough, and so delay sending any data for up to 100ms to see if you can piggy back on the next recv request and reduce the HTTP request count by 1. Also in order to batch up sends. For example:

  Without optimization--
  0ms Long polling Recv
  800ms Send some data
  900ms Long polling Recv (After some data was received)

  With optimization--
  0ms Long polling Recv
      (Send queued for up to 100ms)
  900ms Data sent, in Long polling Recv request.

"There's nothing hacky/quirky about comet... long polling XHR."

Now, unless I'm misinformed, long polling XHR does not work in Internet Explorer, although you do list it as a supported platform: http://wiki.mibbit.com/index.php/Platforms

Could you expand on this at all?

It's absolutely supported.

IE will wait patiently for data to be received, just like other browsers. It'll do keep-alive also with XHR, so you get that benefit as well. You can also remove some HTTP headers in js to get some bandwidth optimization :)

One possibility you may be thinking of is XHR reuse, which is a pain in IE, as far as I remember there's a couple of bugs if you try to reuse the same XHR object. There's hoops to jump through depending on the state or something, and it's just more reliable to not reuse the object on IE.

At the moment, I create a new XHR object for each request, to be sure there aren't issues with IE (I believe XHR reuse isn't an issue on other browsers).

What connection scheduling mechanism are you using? Is it non-blocking asynchronous, or are you using threads? What's the resource overhead per open connection?

non blocking async. Single thread. There's a bit of memory used for buffers, but nothing major. It's been up to around 14k connections total so far in the real world, more in testing.

I think Java is a pretty good choice, especially for doing comet or any other problem where performance is required. Today Java's perfomance almost matches the speed of C or C++... I have some time ago played around with using Jetty's continuations and it's pretty impressive (and highly scaleable).

In my next iteration I'll probably try to go with the long polling model and I'll probably explore C and Java.

Thanks for sharing your architectural notes.

Cool :)

Forgot to also mention CPU usage is negligable. Memory can be a challenge with Java, but often solved by reducing object churn, tidying up better etc. The java.nio package really is pretty awesome.

If it is a poll server, why not write a lite script of lua inside nginx or just a plugin of nginx? That would take full features of nginx and I guess the result would be better.

I think modifying the nginx memcached module would have been ideal. Running a new server that does almost nothing does not seem like a great way to go.

It'd also be interesting to see the whole thing done in Erlang, just for fun.

... or apparently not, for those with little intellectual curiosity about a task quite well suited to Erlang.

What is the advantage of polling over comet? I guess the real question is how often is a user re-polling the same value, without it changing.

-Ben

Polling works out of the box with pretty much all web servers - the downside is that it's significantly slower and more resource intensive than comet.

>> the downside is that it's significantly slower and more resource intensive than comet.

Any data or story to back that argument? Just curious.

I don't have data available right now, but I have done quite a bit of research on server push technologies for one of my projects. It just comes down to bandwidth: regular polling consumes bandwidth (and the related client and server resources) even when there's no actual data being sent.

That architecture diagram makes no sense.

Reply on news.ycombinator.com