Comment by chatmasta | Hacker News Reader

Comment by chatmasta | Hacker News Reader

Parent Full thread

This is why you should run your entire application on bare metal (colo or dedicated) with failover to the cloud.

The benefit of the cloud is its elasticity. You can handle a 2000% traffic increase by spinning up VMs instantly; no need to wait a day or week or month for bare metal provisioning.

And you should do this! But once you've structured your application this way, you are already horizontally scalable. There is no reason the core of your capacity cannot be on bare metal, especially now that container orchestration platforms are bringing feature parity to cloud and bare metal. Only provision VMs for handling temporary traffic spikes.

You can generally predict traffic costs for next 1, 2, or 3 months. You can't predict traffic spikes as easily.

So why not invest in dedicated servers or collocated hardware, just enough to cover your expected traffic? Send any excess capacity, whether due to traffic spike or hardware failure, to dynamically provisioned cloud services. This way you capture the economic advantage of elastic cloud pricing while achieving full utilization of your capex investment.

I've heard this burst-to-cloud story for many years now - Paul Maritz pitched it heavily as the pillar of VMware's value/strategy when he took over there.

I've yet to ever see or hear of it properly executed in practice. I think it's a really hard thing to try and bolt on to a legacy system, and if you build a new ground-up app this way going full cloud to start is easy and cheap for a while until you're successful and then the economics are.. complex.

I think there's also the weird angle to it that the folks who have the skill to architect apps like this are very hard to come by.

Would love to be proven wrong and hear about folks who've seen burst-to-cloud work out in practice though!

> I've yet to ever see or hear of it properly executed in practice.

Isn't Netflix the poster child for this, and haven't they been for a very long time now? Not the content, which resides in their and others CDNs, but the back-end, webservers, databases and management systems are all supposed to be in the cloud now (after a long time migrating).

> I think it's a really hard thing to try and bolt on to a legacy system

There are different levels. Architecting a sharded database solution before you need to might be a bit excessive, but designing for multiple web, storage, or processing servers, etc from the start even if using bare metal is smart, and allows (relatively) easily adding capacity even if you are just colocating somewhere. At the point where you are already able to add capacity by scaling hardware, the cloud is a fairly attractive target, especially since automating the steps you take to add/remove hardware is likely trivial if you've already been doing it manually for a while, as the rough edges have likely been smoothed away somewhat already..

Netflix is not a good example for one big reason: their cost of running servers is dwarfed by their other costs, possibly by two orders of magnitude. Netflix can and does waste money on servers.

At the dollar spend for the infrastructure for most apps, the labor costs of dealing with physical servers is not worth it. Saving $1K/month on infrastructure is meaningless if it costs me $10K in labor costs.

Your numbers only work for the little guy when labor costs are discounted to zero.

Renting a dedicated server requires no labor cost of your own (if a drive fails, your provider replaces it), and is almost always cheaper than the equivalent cloud resource, especially when it comes to bandwidth.

That still leaves sysadmins that you no longer need when using the cloud. The cloud allows your developers to run free without the need for infrastructure and sys admin teams to worry about the servers.

Very little sysadmin needed if you go with a managed dedicated server provider. They pretty much take care of most of that for you. A little more expensive than a non-managed dedicated server but oftentimes still much cheaper than the cloud. Plus, you can't really completely get away from sysadmin when working with the cloud either. You actually get a lot less support than you do with a managed dedicated server provider.

Note: I heavily use AWS and managed dedicated servers.

> That still leaves sysadmins that you no longer need when using the cloud

Looking at our timesheets, managing cloud setups takes more time on average than managing bare metal server setups, and the amount of time spent handling hardware issues make up ~1%-2% of ops time spent on the bare metal setups for our systems. We "save" that on the cloud setups, but end up having spend more to account for dealing with crappy APIs to do all kinds of things we have much more direct control over instead.

You'd have to price in the cognitive load for engineering for that setup, and the recruiting costs / turnover costs for anyone that had to be exposed to the colo center. Also, the colo center! Speed & reliability are features, so you'd have to make sure to pick a colo center that had a competitive setup in terms of peering & backbone speeds, power redundancy, etc. Ideally you'd also be in more than 1 colo unless the cloud is acting as a backup datacenter, so you'd need staff in more than 1 region, lest you fall victim to something like the flooding in NYC and the colocated services that fell down during hurricane sandy.

You'd need to price in continued investment into that colocated hardware too, parts break down, cyclical improvements to hardware happen, etc. It's not simplistic at all, maybe there's money to be saved, but unless your hosting costs are really dragging down profit, I'd pinch pennies elsewhere personally.

> You'd have to price in the cognitive load for engineering for that setup, and the recruiting costs / turnover costs for anyone that had to be exposed to the colo center.

All of which tends to be more expensive if you replace the "colo center" at the end with "cloud provider".

Operating a cloud setup is by no means free of operational issues.

> so you'd need staff in more than 1 region

If you're too small to need that, you use managed hosting, pay a tiny premium over bare metal colo providers, but still far less than a cloud setup.

> It's not simplistic at all

Yes, it is, it is trivial to model, and for companies that don't have the experience, there are consultancies everywhere that will do it for you (that's part of what I do for a living - it's always the easiest jobs to get paid for, because there are hard numbers that shows them how much more they'll be saving that year than what they'll be paying me).

> but unless your hosting costs are really dragging down profit, I'd pinch pennies elsewhere personally.

I regularly deal with clients where hosting costs are measured in tens of percent of total operating costs. For a lot of internet related business, hosting is a massive drag.

Data has weight. You'd need to duplicate some portion of your data between on premise and cloud. I feel the better approach is doing 100% on cloud compute with bare metal CDN / reverse proxy

> Data has weight.

I like this line.

It makes it easier to picture the cost (in time, energy, and architecture) to support it moving from local to the clouds.

That can certainly be cost effective if bandwidth is your main cost. Bandwidth cost is the most atrociously overpriced part of the big public cloud providers, so it's certainly worth pricing out, in particular for people who already are on a cloud provider and where moving would be complex.

Exactly. The beauty of this is that building to scale to cloud as needed makes bare metal even more cost effective compared to cloud, because suddenly you can plan for far higher utilization rate for your fixed hardware than you could before.

One note: you don't even necessarily need your own colocated equipment. If you're in a location with expensive real estate and/or power it is often hard to beat dedicated rented servers for smaller installations, because you often have more flexibility in renting servers further from wherever you have staff than what would work if you colo and need to be able to reach your setup in person.

Rented managed servers still tends to beat cloud with good margins.

Reply on news.ycombinator.com