An engineer’s guide to cloud capacity planning | Hacker News Reader

An engineer’s guide to cloud capacity planning

increment.com

143 pointsgk134 comments

> Perhaps surprisingly for engineers who work in mission-critical business applications, occasional spikes of 90%+ of our users being entirely unable to use the sole application of our company was an entirely acceptable engineering tradeoff versus sizing our capacity against our peak loads.

I think we found the Pokemon Go engineering team.

Acceptable engineering trade off and acceptable business decision trade off are not always the same thing.

That paragraph reads to me “screw the users!” - and for them to say that around “mission critical” seems they don’t understand the critical mission...

The critical mission is to deliver as much business functionality as possible within the business constraints.

The business constraints often include cost. They also may be arbitrarily far from 100% uptime.

Yeah, agreed - but that paragraph was worded in such a way as to convey "screw the users for spiking use, such that the application is now unavailable, and now ALL business halts..." -- Or did I misinterpret that?

The thinking is probably that if usage spikes and some people can't get access to the game and they know that this is because of spiking usage, then this just builds buzz and demand -- like hip restaurants with visible queues.

So in this case the business demand on the engineers was presumably to make sure that 10% of users get good service during the spike while the others are gracefully shown a "Sorry we are too busy right now " message.

Whether or not this is evil depends on the pricing model and other factors that determine whether those bounced user had a reasonable expectation of service.

These engineers and C-levels of this company should be punched in the face.

I just read it as "the compute required to reach 100% uptime would cost more per minute than we lose per minute that we can't access the app." I.e., an entirely business-wise consideration, even if phrased in terms of engineering tradeoffs.

That’s actually a salient point! I appreciate you telling me that as I had not thought of it that way! Seriously.

Depends what happens as to whether it's a problem or not.

Maybe it fails to handle the load, the users get a "failure to authenticate" message or similar, and they try again. If their peaks are really sharp and short, that's probably just fine, because that second attempt almost invariably works.

I think you're parsing that statement wrong: he's saying that engineers who are used to building mission-critical apps would likely be surprised about how acceptable downtime was for their non-mission-critical app (a game).

Well thank god, that makes way more sense!

It's not coincidental; I believe the 'sole application' the author talks about was in fact an online game.

If you are a small startup and do not have strict regulatory or other special req. run on something simple like DO, Vultr etc. and save yourself pain and $.

How do you work on Digital Ocean or some other cloud without VPC? I haven't been able to figure this out. Sure, if I just have 1 or maybe 2 servers it's fine. More than that what are you doing? Mesh networking like ZeroTier? iptables on every machine? It just seems like a lot more work than setting up a VPC.

Not to say it's as robust a strategy as a VPC, but DO does have something called a cloud firewall. It supports the idea of tags that apply to groups of servers.

https://www.digitalocean.com/community/tutorials/an-introduc...

The problem on DO is not outside tradfic but insode. when you enable networking everyone in the same DC network can access open ports on your machines. That elastixsearxh or redis you use needs ssl and pw protection otherwise you have a big problem

You can configure the cloud firewall so that only VMs in your account with a certain tag can communicate, even over the internal network.

Public network + firewall feels dirtier than private network, but in reality they are the same thing: a network that’s only accessible to your VMs, to the extent that you trust the provider’s software to enforce that boundary.

Ovh might be good cheap alternative if you have that need. They have a virtual rack product that's much closer to a VPC.

There's IPsec, although you are now trading VPC costs with sysadmin overhead costs.

A VPN can work very well. tinc [1] for example. Easy to deploy, sustains Gigabit speeds even on weak VMs. And works across different providers.

[1] http://tinc-vpn.org

And is about as good as building network on GRE tunnel, because its homebrew cryptography is written by a dilettante.

if you are running simple web app not having VPC is not a big issue.

This seems more about capacity planning your application when running in the cloud.

I was hoping for some insight when you're hosting your own private cloud, that seems quite a bit more tricky. It's easy to prepare to scale up/down your deployment when all the hardware is already there, but when you're the one running the actual cloud and need to do the same, you can't just do an API call and suddenly have another 100 U of servers and switches racked, cabled and provisioned.

Still a nice article though!

Well if you want to service huge spikes you can run a hybrid env. with on prem. handling base load and cloud absorbing spikes or you have to bite the bullet and overprovision on prem/private cloud based on you specific business req.

So Nutanix and hypergrid are trying to address this, though I’m not sure how well they succeed, but the payment model of hypergrid is enticing: they deploy a cluster for you, and you only pay for what you use, so if you want to have flex scaling you can do that... I’d really look at their offering if you’re looking for a solution for on prem elasticity with a lower cost to enter...

I am not associated with either of these companies in anyway other than looking for a solution to the same problem you mentioned.

I don’t mean to sound flippant here but going Serverless would obviate the need for much of this guide.

I think that you still need capacity planning for "serverless" architectures as well. You need to forecast growth of usage and costs for accounting and budgeting purposes. Does your serverless usage scale linearly with new customers? Or new widgets on the platform? Great, if we're planning on adding 10,000 widgets to the platform next quarter how much will our serverless bill go up? We still need to think about these things.

Also, and this may be my lack of experience, there seems to still be a big requirement for more traditional cloud hosting for databases in a serverless architecture. You have to store that state somewhere.

Going serverless would increase your costs significantly. Have you seen how much AWS charges for Lambda use? Where did this myth of serverless saving you money come from?

I keep hearing this from people who don’t say they’ve built out Serverless stacks. But I keep hearing from people who switched that they’ve been able to cut costs. So in my limited experience this has so far proven not to be true in practice.

Going from autoscaling in EC2 (with a mix or RI for baseline load and spots for peak) to Lambdas has cut our service costs down by half. This is largely due to better utilization rates.

Under-utilized machines in people's stack.

For general compute yes. But you still need to scale your persistent data stores. That is why AWS is investing in things like Aurora. Also you need to understand where to optimize on a cost basis by not just profiling code, but comparing costs, to usage, to performance gains in reduced resources needed.

Serverless means to pay somebody for runs the server.

Serverless is more server and its funny.

Seriously it is a terrible name. It seems to have stuck though.

Reply on news.ycombinator.com