Comment by KaiserPro | Hacker News Reader

Comment by KaiserPro | Hacker News Reader

Parent Full thread

Nothing really, We were asking it to do something its not really supposed to

Basically we needed a request reply semantic. (forgive the explanation) Its where we have an instance of the front end sending a request to any of the back end servers. A single backend replies directly to the same front end instance.

Basically we needed something similar semantic to a load balancer, but in messaging form.

SQS is only a 1-n Queue system. So we can deliver a message to a single backend in a cluster, but we couldn't get it back to the producer without making a message routing system. We used Dynamodb and repeated polling, but that wasn't cheap to scale.

We might have been able use SNS as a broadcast bus of somesort, but that seemed like a backwards step.

We still use SQS for virtually everything else. NATS is kept for that one use case.

> Basically we needed something similar semantic to a load balancer, but in messaging form.

Why not stick with a load balancer? I've been struggling to find a good use case for request-response over message queues so I'm curious.

Let me try and draw a diagram:

Api-gateway -> Lambda -> NATS -> docker backend on GPU

The backend is C++ and needs to be shielded as much as possible from the web. It also takes a fair wedge of time to deal with requests (1-5 seconds) THe rationale is the Lambda front end validates and routes, the backend replies.

Before we moved to NATs we had the added bonus that using SQS meant that there was no possible link between front-backend apart from SQS. (both lambda and SQS are outside of the VPC by default)

the backend has a strict schema, and to keep moving parts down it seemed wise to avoid having to plumb in a webserver as well. They really are badly suited to this sort of thing. They also add a boat load of extra latency.

We have a large number of machines, all of which deal with certain areas, This allows us to intelligently cache hotter areas closer to clients.

Request reply is a reasonable pattern for interacting with backends that are constantly changing capacity based on demand. It also, if done correctly can be a way to optimise for latency over capacity (depending on how you do it.)

With NATS you can ask for more than one response, which might also be useful. Some other systems use a "fastest reply wins" which guarantees a fast response at the expense of capacity.

Reply on news.ycombinator.com