Here we go again.
If you have lots of spare CPU on machines in a web server farm, you have too many machines. Even though I/O is the bottleneck for any single request's response time, CPU is generally the bottleneck on total load.
If you have lots of spare CPU on machines in a web server farm, you have too many machines. Even though I/O is the bottleneck for any single request's response time, CPU is generally the bottleneck on total load.
Maybe we could have switched to something lighter, but the cost of all that development and testing would outweigh a handful of extra machines.
and once you start maxing out boxes, the load comes even harder since the site loads slower. you very quickly run out of resources and suddenly you're SOL and everybody's mad that the site won't load.
honestly, it's hard to point at any one source of problems when scaling a site. cpu, ram, and i/o are all equally problematic. and if you under plan any one of those it can take your site down into a spiral of doom.
you really must allocate your resources for peak load, not for average or nominal load. and then allow for some failures.
And of course, graceful failure is an art in itself. You have to start turning down requests before you actually hit peak capacity, or you risk getting stuck in a thrashing pit.