Hacker News Reader

Comment by rdw | Hacker News Reader

This bit jumped out at me: "200 API servers ... to serve 3000 requests per second". That's only 15 RPS per server. Is that normal for Rails?

View on HN

Rails can serve data far in excess of 15RPS. I've built apps that serve plenty of responses in <15ms. That's one app server on one CPU core. Of course, if you do a lot in a given request it's pretty easy to get that number down as low as you want.

Those numbers kinda surprised me too.

We serve millions of requests per day and have some slow responses (75+ ms) but on any given day our servers handle 175 requests per second without breaking a sweat. =/

Keep in mind that we're a platform, not a website, and we have very little control over things like schemas, slow queries, inefficient looping requests that our developers send us. We can't optimize 500 thousand apps' queries for them.

So for example if an app does something bad like performing a full table scan on every request against a 300 million collection, so every request to that backend is timing out at 30 seconds, and there are thousands of them per second, well -- pretty soon your fixed pool will be full of requests timing out to that backend.

This is totally a great explanation, thanks! This sort of thing is why I hate working on platforms (but also why it's so in-demand).

(we can, and do, do a lot of things to limit the impact any one app can have ... but there are limits. Async is much better for our API model.)

This should pretty much be able scale with RAM and processor using a preforking web server, you are effectively running many copies of your application and the web server is routing them.

Many organizations scale out too soon or for the wrong reasons, and some just have ineffecient database queries and other things that result in bad request times that they could also optimize -- which helps the user regardless of scale with faster page loads.

we had to way overprovision to handle even momentary spikes in availability from any backing store. we aimed to run at around 20% unicorn worker utilization under normal conditions.

But doesn't this sound less about over provisioning and more about optimizing the code you already had?

optimizing won't help you here, unfortunately. the process-per-request model is fundamentally flawed past a certain scaling point.

Jumped out at me too. I did contract work for 2 years helping the API team that powers the bible app (bible.com), and they peak at over 5,000+ RPS on Sunday morning and never dropped below 2,000 RPS 24/7. Their stack was an old Kohana 2 PHP app run on about 18 physical servers at Softlayer. It boggles my mind that you would need 200 servers for this.

Reply on news.ycombinator.com