Well you can't match what AWS provides. Almost nobody strictly can. But it's over-engineered and more than what most businesses could ever possibly need or want. Some questions, does your English only web startup really need redundancy in 30+ countries? Do you really need five 9 reliability or can you get by just fine with four 9s? etc.
For what 95% of the people here on HN need, 2 or 3 decent servers running off of a business FiOS line will be more than adequate for even their wildest ambitions. 99% of the remainder could make do with a rack or two of equipment, a generator, dedicated a/c and some loving electrician work. And even then if they need quick surge capacity expansion, they can still bring in AWS to augment their home-rolled stuff.
If keeping your stuff in-house isn't sexy enough, colo'ing it is still cheaper than AWS in most cases.
There's probably a fraction of 1% of the startups here that actually need what AWS offers. But the prevailing attitude here is that everybody's appointment reminder photo sharing web app is going to require AWS when they'd be perfectly well serviced, and be far more responsible with investor money, with something far less capable. The problem, as I'm seeing in the comments here, is that basic industry know-how (like how to assembly a reliable high-performance server from parts) seems to be largely absent in the industry.