Obviously, it's easy to say that when you're on the bench. Congratulations on the launch by the way.
At my previous firm we had this culture that whenever traffic peaks we spin up new instances. And tools like RightScale & Chef make it ridiculously simple. So our style was to do that than to optimize strains in code paths. Because this is so so convenient.
And before you know it, this notion of hardware is cheap becomes a culture. Soon enough if you grow you’ll be serving 100K users with 250 machines.
I do agree that Chef [and RightScale? Never used it) makes it easy to spawn new instances and through load balancing average your load.
I was talking in term of tradeoff in the first few weeks of a new service with a MVP. Obviously, you re-assess your need before you get to 100k user, and probably uses something else than EC-2.
In any case, both ideas are equally good. I don't claim better knowledge in any way.
If it was better for your previous firm to pay more than to optimize, it's actually preferring developers time (which cost money, as you know) over servers cost.
This can be cost effective until some point. I don't think it could get to the level of "100k users on 250 servers", and if it does.
If the other side of the coin is that you waste dev time AND that your site is down for a few hours.. Is it really worth the "culture fear"?
Also, in some instances of runaway memory, there will always be a point where all the memory in the world isn't enough.