> We're all assuming content comes in constantly, but that's not the case. I imagine they spike heavily at peak times, perhaps 5-6 fold (judging by my own site statistics). Serverless might be good for them because they can handle small loads with small costs and large loads with no effort.

No. I've pushed 1000 r/s (approx) on a price tracking tool at work for keeping track of competitors on one of those boxes. Their largest spike (even if you assume its 10x their average) would be at most 15% of that.

I've personally handled hundreds of requests per second using Django behind Nginx on a m3.medium EC2 instance. The trick to scaling like this is to take full advantage of caching, and the cores available to you. With Django (and Node), this means spinning up multiple instances on a single server (one per available core).

With even moderate caching (say, 1m), you can saturate your VM's bandwidth before running out of CPU or memory.

Ultimately, you will always have a bottleneck. All you can do is identify the bottleneck (via load testing) and address it. If it's network (and you're in Amazon), you have to scale vertically until they remove that limit, or horizontally if you're already on unrestricted instances. If it's the CPU, you'll need to be more aggressive at caching completed computations, or work in a language with less overhead. If it's memory, start evaluating the tradeoffs between CPU and memory. If it's your DB, start investigating why it's slow (it's probably a poor index or configuration; DBs on decent hardware can handle millions of requests per second before degrading).

I don't have any book recommendations; I just recommend creating something and find its limits with a simple load test tool. Then figure out your bottleneck and fix that. Rinse and repeat, and learn.

Its too application centric to be honest to give good generics unless you've built something similar before.

> As a concrete example, say you have an ecommerce app. Read requests take 80ms while write takes 200ms.

An ecommerce site can be nearly 100% cached except for cart/session/search data. This gives it a massive advantage over everything else (since you can load the non-cached data via js and its relatively tiny).

I think the last time I benchmarked one of our front end servers it was ~500 r/s on 2 cores for our search page.

You really aren't going to know until you try it and benchmark it yourself tho.