Hacker News Reader

Comment by HALtheWise | Hacker News Reader

I was assuming that it's option 2 (you can spin up new workers within 10 days). If you're using cloud compute this is almost always true.

If you're building an on-prem cluster, I'm assuming you either spin up cloud workers for the extra load, have extra servers that are fully powered off and can be booted in a few minutes, or just physically order and install new hardware. Amazon can ship you a computer in 2 days, so it's not inconceivable to design an on-prem infrastructure to allow significant scaling with short notice.

Option 3 is sorta always true as well, since there's always compute load at your company that's lower priority, and can be shed in an emergency (for example, your CI system).

In any case, I suspect the common pattern of scaling is that most companies grow in a smooth enough way that they can predict demand in future weeks within a couple percent, so you can just run at 98% CPU instead of 100%

View on HN

The idea that you can run an on-premise cluster and just supplement it with cloud workers is a nice idea in theory. But the reality is that this never happens.

Because the two main reasons that a company would be running an on-premise cluster is (a) security and (b) cost. Both of which conflict with running a hybrid model because it is extremely complex and expensive to do so.

Perhaps today this is true.

In 2007 I was working for a webhostet / nascent iaas provider, and we had workloads doing exactly that.

Specific example would be Celtic FC who had a baseline of dedicated, but would scale into our VMs during events, i.e. uefa cup games.

Reaching into on-prem hardware from the cloud is a substantial engineering effort, and one that can land you in the news if you do it wrong.

Good points.

For security concerns, you would probably be selective about which services and workloads you move to cloud compute, possibly preferring to offload internal services like CI before customer data.

Regarding cost, are you implying that there's large monetary cost to preserving the ability to spin up cloud workers in an emergency, or just that once you do it's expensive per day to keep them running? Presumably if you're running an on-prem cluster for cost reasons, you would work to scale it up quickly after an unexpected sustained demand spike, and the cloud supplement would be temporary. If you're getting "unexpected" spikes every week, you need to work harder at forecasting.

In any case, I suspect today that people running on-prem clusters provision them with the intent of being able to handle a load growth and keep weekly average utilization at <60% during the spike. Investing in building infrastructure that keeps working smoothly when weekly average load is 90-100% means you can provision a smaller cluster either way.

“Never” is way too strong.

This technique is called “cloud bursting” and is done often enough to have docs on Azure: https://azure.microsoft.com/en-us/resources/cloud-computing-...

More generally, hybrid on/off prem (or multi-cloud) is a top level feature of GCP’s Anthos.

You’re not wrong about your two main reasons, and it’s true that it’s hard to wire up. But you definitely can wire up hybrid workloads in some cases.

The obvious one would be stateless batch compute like image/video processing; “I prefer to re-encode my videos on prem but can burst to the cloud if I get a traffic spike”. This might end up being cheaper than either of all-on-prem with overcapacity, or all off-prem with auto scaling.

I agree it’s not likely to be useful for something where security is your reason for going on-prem.

Reply on news.ycombinator.com