Hacker News Reader

Comment by zamalek | Hacker News Reader

If your code can survive chaos monkey, it can trivially weather this. Netflix, as one example, could run their entire infrastructure on these VMs.

View on HN

Well except that on peak hours their infrastructure would disappear :)

You always need some traditional instances to serve things that need to be available at all time, like your website. Now you could run things using spot instances and switch to traditional during peak hours but that requires confidence that enough capacity is going to be available when you need it (which is going to be at the same time as everyone else, so it's always a bet :))

Apparently some just gamble that by having some instances of every time, they won't all go down:

"However, your cloud provider can pull these virtual machines out from under you at any time without notice (because someone else is willing to pay more). So how do you lower the probability of losing all of your instances to nearly zero? Cloud providers generally need to have a large buffer of capacity and they have many different virtual machine configurations with different CPU and memory sizes. So, don’t just spin up virtual machines of a single configuration type, spin up lots of them from many different configuration types!"

https://blog.comma.ai/scaling-for-10x-user-growth/

I'm not sure if they still have a fallback of traditional instances.

Yes absolutely. On AWS you can use spot instance fleets to implement this strategy https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/how-spot...

Except that (presumably) there is no guarantee that any spot instances are available. You could wind up in a state where you have no instances running. No user-facing production service can tolerate that.

I believe AWS can create spot autoscaling groups which include a minimum baseline traditional instances plus a floating level of spot instances. This seems to be a nice compromise.

Just spin up regular VMs if you can't get spot instances when you need them.

Lack of spot VM means that all spare capacity has been taken up by the regular VMs, meaning that it spots are gone, so are regular ones.

Having worked on infrastructure deployed this way, I can say this oversimplifies a bit. There's virtually never no spot capacity at all, there's no capacity at a given bid price, for a given instance type, in a given region. You can start to scale up on-demand infrastructure using spot price as a signal, before there's literally none left, and you can also run on clusters of heterogeneous instances, and change your instance type mix when one instance type's price goes up. In the worst case you can fail out of a particular region whose spot prices have gotten too high. Lots of options.

No shade, but this is a very AWS specific view of cloud. Every cloud has a different resource allocation philosophy, not all the assumptions are the same.

Yeah, GCP spot instances don't vary in price like AWS ("The price adjustments will occur at most once a month").

There is no longer bidding for spot prices on AWS. They moved to model where pay the fixed spot price which isn't based on capacity.

https://aws.amazon.com/blogs/compute/new-amazon-ec2-spot-pri...

Interesting! I think it still updates more than once a month, but that is a significant change that I wasn't aware of, thank you.

Maybe that's true but we haven't seen this happen in practice. Maybe we're just incredibly lucky but we're able to get on-demand instances to recover after spot termination.

No guarantee but still very likely there are instances available.

Spot is excess capacity. If there's zero excess capacity, that means there's (probably) also no on-demand capacity as well.

If you are not being given any excess capacity, that doesn't mean there is no excess capacity. You are just not one of the lucky few, whose numbers are shrinking.

Reply on news.ycombinator.com