Hacker News Reader

Comment by jfager | Hacker News Reader

"This is then picked up by every JSON-RPC server and they will intersect those two tags, sending the event {hello: 'world'} to any connection that matches."

Do you see this eventually becoming your bottleneck (every rpc server has to handle every event)? If so, what's the plan for reducing the event fan-out?

View on HN

Yes we do see this as becoming a bottleneck. We are already working on a solution for that. I mention in the "What's Next" section about a bit more advanced routing using ZeroMQ instead of Redis.

The problem is a difficult one to say the least. We're nowhere near the bottleneck now, but we do want to resolve this long before it is close to becoming an issue.

Could you please elaborate on how using ZeroMQ would solve your problem? I have a surprisingly similar setup and I'm also worried that this is going to be a bottleneck.

ZeroMQ doesn't solve the problem. ZeroMQ is flexible enough that we can build a solution. We're working on distributed routing and registration using ZeroMQ and Redis as it's data store.

The RPC servers register the tags and channels we distribute based on that data. Any ZeroMQ server can process and deliver any event to the end RPC server.

We'll probably do a detailed writeup when this is done. We plan on using this for all the infrastructure wiring when it's done.

Please do a writeup when you've done this - would love to read about this - very relevant to my work as well.

Reply on news.ycombinator.com