RPC over AMQP | Hacker News Reader

RPC over AMQP

hydrogen18.com

25 pointshydrogen1818 comments

The real issue is that most of the time is spent marshalling all the data into YAML

Using MessagePack would speed that up significantly, too. I've done a comparison of MessagePack vs. JSON vs. YAML vs. Ruby .marshal in the past[0], and I'm currently building a system using AMQP+MessagePack (as of now the only part that's public is [1]).

Basically, YAML is great for configuration files because of its flexible syntax, but terrible for communication because of its flexible syntax.

[0] http://blog.mikebourgeous.com/2012/10/12/optimizing-message-... (missing numbers for MessagePack, but mentions 2x speed up beyond best other option)

[1] https://github.com/deseretbook/classy_hash (includes a benchmark script that shows how ridiculously fast MessagePack is compared to YAML)

You're absolutely right. The performance and security implications of YAML make it an awful choice.

Unfortunately, upgrading the server component is trivial but upgrading all the clients is next to impossible. The logical next step after this project is to implement a saner set of endpoints that breaks up all the amalgamated functions. At some point in the future, there will come a day in which we'll simply ignore the requests from older clients.

Not only is it faster than the serializations you mentioned, messages generally have a much smaller memory footprint, and it is therefore very friendly for use in contexts where bandwidth is likely to be at a premium (e.g. mobile carrier data limits). I use it exclusively in mobile apps I develop.

Native mobile apps or hybrid?

Native. I haven't worked on any mobile apps that are webapps for mobile, so I haven't used it with the JavaScript implementation.

AMQP is so nice from an engineer's perspective. If shit goes south somewhere in the system, I get a pagerduty alerting me that a deferred queue has exceeded some depth and I can start working to resolve the issue (if a message fails to be processed it is sent to the deferred queue). Once the issue is resolved, all the messages are still waiting in the deferred queue and will get processed as soon as they are requeued, and I don't have to replay nginx logs. Anything that can be asynchronous, make asynchronous and save yourself the headache of dealing with 500s when a component of your distributed system fails.

This. Maybe one of the top reason to use queues and ESBs

So to speed up a request, the author wants to introduce a whole message queue, and rewrite it in another language. This seems like a big maintenance burden to me.

Could the request be changed to use a faster message format instead of YAML?

If YAML can't be replaced, could the YAML marshalling be sped up?

> If YAML can't be replaced, could the YAML marshalling be sped up?

For sure... for example, by rewriting it in another language?

You could just use a marshalling lib written in a faster language and bind it to your current language and code instead of rewritting everything

I'm a little curious why in this case one wouldn't choose to do "RPC over HTTP".

Assuming the following points from the post are correct:

a) the perf problem is YAML marshalling b) that can be solved via a go rewrite

then perhaps an easy way to solve "I want to call the old code" is to access it over HTTP, using a cheaper marshalling method than YAML.

I guess the tradeoff is whether you'd prefer an AMQP broker in there mediating the requests or an HTTP load balancer (or http client pool code in the go app). And whether you see other uses for that broker in your setup.

You can make it so that finding a free slot is always O(1). Keep another array that is a stack of available slot indexes. When you need a free slot, pop off an index and use that slot. When you're done with a slot, push its index on the stack.

You're correct. I mentioned that approach near the end of the article.

OpenStack uses RPC over AMQP: https://wiki.openstack.org/wiki/Oslo/Messaging

I implemented RPC over AMQP for the Spring AMQP project, for all those who are interested in using this kind of thing in a JVM environment.

The documentation describes how it works here: http://docs.spring.io/spring-amqp/reference/htmlsingle/#remo...

Why not use zmq (http://zeromq.org/bindings:ruby)? And for data serialization use Google Protobuf's port ruby-protobuf; and avoid introducing a central message broker as another point of failure?

I've used zmq (or is it 0mq) in the past.

My complaint with 0mq is that you wind up needing some sort of service discovery service from what I can tell. In order to get everything working, each client needs to know about at least one servant. Ideally, each client needs to about all servants and use a load balancing algorithm across them. I'd need to keep this configuration at all clients synchronized. I'm aware there are tools to do this, but ultimately I am not the person in charge of operating this.

Nice work. I did something similar for .NET -- see http://www.restbus.org

The project isn't complete but is functional enough to be used in production.

Reply on news.ycombinator.com