Message Queue Evaluation Notes

+1 to this, a lot has changed since this was written.

Second Life involves a lot of per-user message sending. IM, group chats, friend requests, marriage proposals, dialogs with in-game merchant systems, payment notifications, persistent announcements, and a lot that I'm forgetting. They're all roughly the same thing -- messages that need to be delivered to either a specific user or a group of users, with flags for whether they get discarded when the user is offline or whether the user has to explicitly acknowledge them.

It was probably overreaching for us to try to find a system that would handle all this for every user. But even hundreds of thousands of current users, the bare minimum we could expect to be concurrent, were not well supported by the existing solutions.

I'm the primary author (formerly Which Linden). Surprised to see this pop up here after all this time!

We wrote this almost three years ago, and at the time ZeroMQ was substantially less mature than it is now. In particular, the Python bindings were broken, weren't maintained by the ZeroMQ organization, and there was no indication they'd be production-ready by the time we needed it (which would have been within the next month or two).

More importantly, we were looking for something that handled our use case a little bit more fully. Our message-rate requirements were laughably small, so what we really wanted was software that gave us an "inbox" for each of millions of users. We also weren't looking forward to implementing failover, consistency, and sharding from scratch.

Edit: corrected timeline.

there is seriously something to be said for the complete solution that other systems provide.

if only as stepping stones towards eventually using something like zmq.