You conveniently forgot a key bottleneck: the amount of RAM on your server. That's what determines how many concurrent sessions you can keep running before performance degrades and you need to add another server.
If anything, I work hard to put more stuff in RAM to avoid hitting disk, as even though we have SSD's in all our new servers, disk IO is still inevitably where we hit limits first, followed by CPU. Running out of memory is a rare event, and when it is it is consistently an indication of a bug.
I don't understand. Concurrent sessions? Do you mean open connections? Worker threads? What exactly?
The maximum speed of the internet, the maximum speed of your database, the maximum speed of your programming language are all basically constants that you can't improve by throwing money at the problem.
As a programmer, available memory is absolutely your domain or else you're increasing hardware/VM costs for yourself or your employer needlessly. Then you've transferred the cost of increased hardware to host your app/site/whatever off to users.
I can see today bit pinching isn't fashionable as it was long ago when memory wasn't cheap (it's not really "cheap" today either since you're paying proportionally greater per VM than the actual cost), but when a basic tenet like efficiency is ignored because DB all you're doing is compounding the problem.
BTW. Most people already deal with the DB issues with aggressive caching and/or reverse proxy so that still leaves the core application.
In general if your server is making you money, you'll make more money improving the service than reducing the amount of RAM the service takes to run.
In summary, test the conversion rate of a page/feature/etc, not how much RAM it uses. If you're going to performance optimize, attach a profiler and take the easy wins, don't spend too much time on it unless it's a ridiculous amount of resources or your margins on razor thin.
That's not a law of physics. The number of servers you need depends on the number of users you have. The functionality you have to build usually doesn't. So the more users you have, the less true your statement becomes.
I don't think that has anything to do with the languages. I think that has everything to do with the quality of the memory manager implementation, and there is at least one memory manager that does deliver in this respect, namely C4.
There are times when the benefits are greater, for example when your software is running a million instances, or you're working on a hardware-intensive game, but those are certainly not common case.
Once you get past a certain level, though, the cost of the next 1GB isn't $20, it's $20 plus the cost of another computer plus the cost of exploiting multiple machines rather than just running on a single one.
Then it's $20/GB for a while again, then $20 plus the cost of adding another machine, and at a certain point you need to add the cost of dealing with the fact that performance isn't scaling linearly with amount of hardware any more.
That last bit might be a concern only in fairly rare cases. But the first, where you make the transition from needing one machine to get the job done to needing more than one, isn't so rare. And that can be a big, big cost.
(Very similar considerations apply to CPU time, of course. Typically more so.)
This casual disregard for resources would explain why a lot of startups run into infrastructure issues so quickly and settle on terrible solutions (or outright snake oil) to mitigate problems that shouldn't exist in the first place.
People need to start realizing that The Cloud is only a metaphor. Hardware isn't intangible and neither is their cost.
Servers don't grow on trees; they live on a rack and grow on electricity and NAND.
It's easy to think everyone has their acts together like facebook or google, but most companies i've dealt with have hardware upgrade processes measured in months or years, not hours or days. You absolutely have to take responsibility for your work as a programmer and make stuff run fast instead of labeling it somebody else's problem.
Well... For those companies, obviously, dynamic languages are not an option. ;-)
I remember trying to optimize some financial software a couple jobs ago and hitting a brick wall because that's the speed the disk rotates at. We ended up buying an iRAM (battery-backed RAM disk) and sticking the DB on it. You can get this a lot cheaper by avoiding the DB and using a RAM-based architecture if you're willing to sacrifice fault-tolerance under power outages (or if you have some other architectural solution for fault-tolerance, like writing to multiple computers).