Hacker News Reader

Comment by Oculus | Hacker News Reader

Took a look at your code and noticed you're using Socket.io. Don't use Socket.io for anything over double digit concurrent users. It's well known to be horrible at scaling. If you want something that's better at scaling I'd suggest SockJS[1].

1 - https://github.com/sockjs/sockjs-node

View on HN

The "over double digit" claim is simply not true. I'm running three multiplayer games for about two years now, and it starts to have problems once if goes over 3500 concurrent connections. It could be that I just hit the right combination of node+socketio versions, though, as I had problems with newer ones. My setup is:

node.js 0.6 socket.io 0.9.6

This version of socket.io has a nasty bug in jsonp protocol handling which I patched manually by merging some change from a newer version, but it does work fine.

To make it scale, I run multiple node processes. In the last 300+ days each of those has between 1000 and 2000 concurrent connections all the time and it works fine. Except that there is some memory leak (I'm not sure if it's related to socket.io or some other lib), so I restart each server once a week (it's done automatically each Sunday at 2am).

I had my eye on SockJS since 2012, and it looks like newer versions are really good, but I'd need upgrade to newer nodejs and this is simply not possible for the existing projects as many used modules are not available.

However, if you're starting a new project, it's worth testing and see which performs better.

We also do had a memory leak because of socket.io 0.9.6

take a look at this discussion https://github.com/LearnBoost/socket.io/issues/1303#issuecom...

Interesting, but I'm not using Redis nor MemoryStore. I gave each machine behind HAProxy a different hostname (mapped to the same IP address), and I route the clients in roundrobin fashion to each of those before they open the page with socket.io javascript. It works like this:

- client opens a page on HAProxy load-balancer picked server

- the actual node server that receives the request, redirects the browser to its custom hostname using the same URI

- client opens the same page on the particular server and socket.io connects to that subdomain

This ensures that all future requests (if socket.io connection is broken) will go to the same server without Redis, MemoryStore, cookies or whatever.

Drawbacks:

- one additional HTTP redirect when page that contains socket.io is loaded (luckly, in games there aren't many of those)

- if you want to retire some sub-domain hostname, you have to be careful to allow existing clients to re-connect somewhere else. Not really a problem for my setup, but YMMV.

How does this work for non-socket transports? You have to be saving the broadcasts in case someone is polling, right?

I'm not sure what you mean by "saving the broadcasts". Can you give an example?

Some 11% of clients are using XHR or JSONP protocols and it it Just Works(tm), I did not add anything special.

You said you don't use Redis/MemoryStore, but when you broadcast something it has to be saved somewhere until all of the polling sessions get it. Where is it being saved?

There's a memory leak that only got fixed in 0.10.22 I believe. Erin Hammer and the whole Node Black Friday team really publicized it when it was biting them[1]. I'm not sure if the leak was present in 0.6 and given that you rely on older modules the cost/benefit of switching is probably too high.

You're the first person I've heard that was able to run vanilla Socket.io at such scale. I know Trello and some other shops use Socket.io, but they've forked & changed it to something that looks nothing like the original[2].

1 - http://www.joyent.com/blog/walmart-node-js-memory-leak

2 - http://nodeup.com/fortytwo

Seconded. I ran a very popular website that used Socket.IO. It broke down whenever we had 200+ concurrent users. Sockets would randomly close.

Anything to backup your claim about Socket.io?

This epic issue: https://github.com/learnboost/socket.io/issues/438

Great podcast about using Websockets (including Socket.io) in Node: http://nodeup.com/fortytwo

"Under the hood SockJS tries to use native WebSockets first. If that fails it can use a variety of browser-specific transport protocols and presents them through WebSocket-like abstractions." - SockJS

How this is helping?

"Flash is absolutely not required for Socket.IO to function. If Flash is available, it'll be leveraged, as it provides almost the same capabilities as WebSocket. If it's not, the next best transport will be chosen." - Socket.io

My understanding is the issue is not in the protocols, but in the implementation of the libraries themselves.

This is a rather inaccurate statement to make. I've used Socket.io with double digit concurrent users before and Socket.io handled things fine.

You're not disagreeing with the parent. They said more than double-digit.

I'll look into it. Thanks

Your app's down :(

i'd suggest https://github.com/primus/primus. it's an abstraction layer for real-time so no lock-in to a single socket implementation. it's simple and extensible by plugins.

Reply on news.ycombinator.com