Comment by BMorearty | Hacker News Reader

Comment by BMorearty | Hacker News Reader

This gem had a huge impact in May of this year on one of Airbnb’s Ruby services. Last year when I was still at Airbnb I did the research and foundational work to make Airbnb’s HTTP client gem compatible with the Async gem and Falcon server. I’m no longer there but this year, one of my friends who’s still at Airbnb put that work to use on Airbnb’s Ruby service that talks to a Genesys API.

Most Genesys API calls have reasonable response times but there are these occasional, weird, unpredictable 30-second response times. The slow responses can come in bursts, where you can a bunch of them at once. This makes it extremely hard to predictably scale the Ruby service that calls it, because in normal synchronous mode, every thread that calls Genesys is blocked and on longer available to take more client requests. You could scale up the service to assume these 30-second response times will happen all the time, but then you’ve got huge server costs just for the 1% case.

But using Async and Falcon (an Async-compatible HTTP server), Airbnb was able to safely scale down the service by around 85% and turn on autoscaling, and it’s working beautifully.

I would love to learn a little more here. So for 30 second tail latency cases, doesn't Ruby HTTP client support a timeout with cancellation? The only useful case where I believe Async Ruby can help is the parallel requests cases.

Disclaimer: I am not a full time Ruby person, so if it sounds naive please feel free to point it out.

Ruby’s built in timeout library creates another thread with a sleep call. If something is blocking on a socket in the C code, the timeout doesn’t work.

Right, don’t use that, but there are well-implemented timeouts in network libraries, etc.

The fiber scheduler also hooks into the `Timeout` module to provide safe(r) timeouts using the event loop.

Is it true that a timeout wont' work on something blocking on a socket in C code? As I understand it, normally, a thread blocking on IO -- even in C code -- does not prevent context switching to another thread -- such as the timeout thread.

But there are other reasons to avoid the stdlib timeout library, it's true; and also other ways to timeout on a network call, network call API's should offer their own timeout arguments normally.

Yes, Ruby’s HTTP client supports timeout. As I mentioned, we weren’t using Ruby’s HTTP client, we were using Airbnb’s HTTP client with some additional capabilities. But it also supports timeout. The issue here was that I don’t think it would help to time out with cancellation and then try again (if that’s what you’re suggesting). Genesys API calls weren’t failing, they were just returning very slowly. A retry wasn’t necessarily likely to return any faster since the slow responses tended to happen in bursts, presumably based on something weird happening on their side. Meanwhile, phone calls continued coming in to Airbnb’s support center (this gem handles incoming calls) and we needed to continue routing those to Genesys as well.

You mentioned “The only useful case where I believe Async Ruby can help is the parallel requests cases.” That’s what was happening here: because of the tail latency, we sometimes needed to have a lot more parallel requests.

Does that clear it up?

How does falcon come into the picture here? Is this structured like a microservice between rest of airbnb and genesys, so that whatever needs to interact with genesys, interacts with this service instead and get a consistent latency?

The Airbnb service’s API is implemented over HTTPS. Falcon is built on top of Async. As its README says, “Each request is executed within a lightweight fiber and can block on up-stream requests without stalling the entire server process.” So whenever an API request comes into Airbnb’s service, Falcon creates a fiber to handle it in-process. If that fiber makes an API call to Genesys, control returns immediately to Falcon in the main fiber’s event loop to be able to handle more requests while the request fiber is awaiting Genesys’ response.

I don’t know the details of where this service sits in the service diagram. I think it may actually be something Genesys calls, and it in turn sometimes calls back to Genesys. No, the point of this service is not to get consistent latency on Genesys API calls.

Reply on news.ycombinator.com