I agree with your sentiments, but this:
> I always respond: "And why do you believe that you would do any better?"
…is an apples-to-oranges comparison that implies that the thing that Amazon is currently doing is what you would be doing in an Amazon-less scenario. It's not.
Amazon hosts thousands of customers and needs to service all of them in the same infrastructure, which coincidentally is also shared between every customer by virtue of being virtualized. It's a complex structure in which every resource -- CPU, RAM, disk, network -- is shared, which means that even a minor EBS problem could potentially have a butterfly-effect propagating through multiple customer's hosts. There are limitations imposed by the structure; for example, the only way to access large amounts of block storage with local-like performance is through EBS. But EBS has been shown to be high-latency and flaky, and Reddit's chronic problems with EBS is a good example why it one probably should stay away from it entirely. Network latency on EC2 is also pretty horrendous compared to classic non-virtual setups; I have set up HAProxy clusters where the connection setup time to backend hosts would be measured in tens of milliseconds.
Now, if you hosted your own stuff, you would be the only customer, and you would not be sharing resources with anyone else, and you could design your hardware exactly to your specifications (fast local disks with low latency, fast networking with low latency and so on). The difference in complexity is significant. There are tons of challenges and costs involved in hosting your own stuff and maintaining uptime, but the complexity equation is different.