If you enable Multi-AZ for RDS, your bill doubles until you cancel. If you set up two servers in two DCs, your initial bill doubles from the CapEx, and then a very small percentage of your OpEx goes up every month for the hosting. You very, very quickly make this back compared to cloud.
Disclaimer: I work on a cloud networking product.
Not to mention setting up and maintaining your physical network as the number of physical hosts you’re running scales.
In the locations where there's more traffic, and we need more servers, there are more cost effective providers, but there's value in consistency.
Elasticity is nice too, we doubled our instance count for the holidays, and will return to normal in January. And our deployment style starts a whole new cluster, moves traffic, then shuts down the old cluster. If we were on owned hardware, adding extra capacity for the holidays would be trickier, and we'd have to have a more sensible deployment method. And the minimum service deployment size would probably not be a little quad processor box with 2GB ram.
Using cloud for the lower traffic locations and a cost effective service for the high traffic locations would probably save a bunch of money, but add a lot of deployment pain. And a) it's not my decision and b) the cost difference doesn't seem to be quite enough to justify the pain at our traffic levels. But if someone wants to make a much lower margin, much simpler service with lots of locations and good connectivity, be sure to post about it. But, I think the big clouds have an advantage in geographic expansion, because their other businesses can provide capital and justification to build out, and high margins at other locations help cross subsidize new locations when they start.
From outside it seems that either AWS picked the wrong default to present their customers, or that it's unreasonably expensive and it drives everyone into the in-depth handling to try to keep cloud costs down.
- https://news.ycombinator.com/item?id=29473630: (2021-12-07) AWS us-east-1 outage
- https://news.ycombinator.com/item?id=29648286: (2021-12-22) Tell HN: AWS appears to be down again
Maybe things are better now, but it became apparent that people might be misusing cloud providers or betting that things work flawlessly even if they completely ignore AZs.
After that, yeah we’ll let AWS do the hard work of enabling redundancy for us.