Hacker News Reader

Comment by bklyn11201 | Hacker News Reader

Even with the discounts, savings plans, multi-year commit discounts, etc., the cloud costs are absolutely absurd if you are running stable applications where you understand the future growth, use, and demand. Yes, the cloud is the right place for 99% of apps out there, but the market is sending a clear objective out to all tech companies: become profitable and build a sustainable, high margin on your products. Every maturing company with predictable products must be exploring ways to move workloads out of the cloud. AWS took your margin and isn't giving it back.

https://a16z.com/2021/05/27/cost-of-cloud-paradox-market-cap...

View on HN

> Even with the discounts, savings plans, multi-year commit discounts, etc., the cloud costs are absolutely absurd if you are running stable applications where you understand the future growth, use, and demand.

This is true, but it doesn't preclude use of the cloud.

The core of the cost issue is, in my view, that software is still designed to run in a datacenter: always on, all code in a single process (yes, this includes microservices), expected to never stop mid-execution. That made sense when you owned the hardware and needed to fully utilize it for the duration of its life. This prevents the best cost-saving measure in the cloud: turning it all off.

People can complain all they want about how the cloud providers operate, but there are ways to utilize cloud infrastructure without breaking the bank. Software development practices have to change, and in the meantime the cloud platform providers are going to make a pile of money as companies shift legacy software to the cloud without the needed re-architecture. We need frameworks and other technologies that, similar to the abstractions we now have for memory management, multi-threading, asynchronous I/O, etc. relieve the burden of reasoning around distributed, disjoint processes from the developer. Then we can finally realize the ideal of "only pay for what you need".

>all code in a single process

Can you give some example of that being the case?

> This prevents the best cost-saving measure in the cloud: turning it all off.

However, the cost calculation is weird when you factor that 2 hours of cloud "on" can often pay for an entire day of "on" for a normal server.

All else being equal, which I'm aware is a contentious/controversial notion in of itself, even if I personally believe the colo costs for headcount are often greatly overestimated.

Nice try.

">People can complain all they want about how the cloud providers operate, but there are ways to utilize cloud infrastructure without breaking the bank"

Alternatively do what I do (example of particular product I developed and nor run for a client):

Single exe in C++ talking to a local Postgresql database. Combo is capable of processing thousands of requests per second sustainably without breaking sweat (this will cover business needs for a next 20 years). Also standby always on server with replication. Daily backup to on premises backup server. Hardware for production and standby is dedicated 16 Core AMD with 128GB RAM and 2x4TB SSD. Each rented from Hetzner at about $100 each. Fronted by Clouflare, not sure about the price as Client pays it. All deployment / management / maintenance / backup / Point In Time Recovery is done by couple of scripts. Reinstalling whole thing on a totally new server takes running single script for about few minutes (at some point will take longer as the size of the database grows but would still be very manageable).

The cost on Azure for the equivalent processing power and features accordingly to their calculator is 10 times more.

This is great. Well most companies I migrate to the cloud are not running a single workload that fits on a single server.

Most companies that save a lot by moving to the cloud are the ones which have several workloads, multiple products, teams, and verticals. They got security requirements, compliance, logging requirements to name a few.

So yeah, if you only consider a most basic business that has a single website with a single use case and non of the above then Hetzner is a great option.

This is why there are containers and you can rent multiple servers as well if there is a need. Security requirements have nothing to do with being on the cloud. You just basically have to encrypt data at rest and satisfy couple of other things to get your SOCS2 label. Did it, no sweat.

>> but there are ways to utilize cloud infrastructure without breaking the bank.

You seemingly refer to a discounted mode of execution in AWS where your servers can be pulled off at any moment and there is no uptime guarantee. That only works because few use this mode. If everyone starts using it, the discount will disappear.

I suspect they were mainly talking about "serverless" (Lambda, Cloud Run, etc) and managed services vs. running your own compute in the cloud.

Whether it can be made cost-effective in practice or not, I don't know - I haven't made the calculations. Just saying that's how I read it.

I haven't see that work out in practice, I've suspected that's mostly marketing fluff at this point. Anyone have counter examples?

We considered autoscaling web backends once. You've got to worry about capacity issues. You don't want a load spike only to discover EC2 has run out of the instance type you use. I've seen that issue 3-4 times in AWS now and it's painful when it causes downtime. Your app needs to have spikey load for the pricing to work, for us, even a strong day/night cycle didn't give enough "off time" to cover the higher hourly prices. Cloud vendors give steep discounts for always on usage. Your load also can't be too spikey, as spinning nodes up and down takes time. Making nodes start quickly can be a dedicated project.

I think we also looked into moving CI/CD servers to on demand, but some engineers worked too late into the night and others were up early. IIRC the "off period" was long enough to provide some cost savings, but not high enough to justify engineering cost.

Something like Lambda doing a "scale to zero" with a generous free tier looks cheap, but if you ever need to scale up you'll find it's quite expensive and your stuck with a weird architecture that's hard to move to something traditional. It is cool when an app fits the Lambda free tier, but that's not a good model for companies paying very large cloud bills.

The problem is that someone needs to do capacity planning, someone needs to buy servers. AWS has to pay the cost of an unutilized server by... keeping the hourly cost of on demand servers/cloud functions high.

Dropbox (from the a16z article) to me is the exception that proves the point.

What percentage of COGS is your cloud/infstracture spend? If your products don't look like cloud infrastructure (aka Dropbox), it better be very small. If you think cloud is costing you 1000% what it would cost on-premise, that's almost certainly a spend and architectural discipline problem, not a cloud problem.

Hum... A 1000% overhead looks typical to me.

Whether that's an important saving or not changes from business to business, but anything less than 1000% seems only achievable by customized discounts from the cloud provider.

I feel like this is comparing some cost-optimized on-prem scenario with a "idgaf, I'm prioritizing growth over cost savings" cloud scenario. Every company I've worked at has been way more concerned with growth to the point that the cloud spend is wildly careless, but we could easily reduce it by a factor of 10 with dramatically less effort than it would take to move on prem.

Is saving 50% also a worthwhile goal at expense of being on prem, and all the downsides? Asking honestly because that’s where my business is in terms of the decision. We won’t save 90% but we will save 45-55%.

Is your company experiencing rocketship growth or are you at the point where your growth has stabilized to a modest figure? It's probably best to pursue growth over cost savings, if possible. But not every company is in a position where a few more features can double revenue.

Still, I think it's hard to justify going anti-cloud in an effort to cut costs. Without a very skilled and experienced team doing the migration, the end result is very likely to be some hybrid setup that's the worst of both worlds. Especially if you're using a managed service, the open source alternatives might not be the drop in replacement that they claim to be.

Is your spend 2% of revenue? 5%? 50%? Saving 50% of 2% is almost certainly not worth the risk. The bigger your the percent and your revenue the more likely you should do it. Dropbox again is a good example.

There are a lot of variables, but unless the 50% savings isn't much more than enough to hire a very good ops team, then no, I can't imagine that is enough savings to make the trade-off worth it.

You get the added benefit of a very good ops team. A very good ops team is typically the difference between a good and bad IT company.

Remember that the choice isn't simply between AWS and racking your own gear. You can make big savings by just renting Linux servers from hetzner or similar. They install and repair the HW, you get ssh access just like with a cloud VM. You just have to make sure to monitor it so if there's a failure you can file a ticket to get components swapped out.

Which is really strange to me why uber just recently announced they are moving everything to the cloud.

If you're a big company you can easily negotiate "normal" rates.

Speaking from a dev perspective: I prefer the cloud. There's less red tape to getting infrastructure provisioned if you work on a team that's outside of core product. Managed services give you access to serious compute capacity by filling out a 5 minute wizard.

Plus, most jobs today expect cloud experience. So there's an incentive for developers to push to use a cloud so they can grow into their next role.

> There's less red tape to getting infrastructure provisioned if you work on a team that's outside of core product.

This here is the compelling case for the cloud. It was always less about the features and the money save, it's that I now have 1 stop shop vendor, if I need a queue I can spin it up in 10 minutes, instead of filling out a ticket to internal IT getting them to put it on their backlog waiting until they have the cycles, and in the meantime I'm sitting on my thumbs.

I need a new server, in the old times it was a ticket, a budget meeting, two or three levels of sign off, a business case, on and on. Whereas in AWS or Azure I can spin up a box in about 20 minutes and be all ready to go, and happily start developing.

Just like MS was never about technical excellence, it was about being a pre-approved vendor that had everything you needed, so you wouldn't have to go through another vendor approval process. Same with the cloud now scaled down to a team size, I now know I am pre-authorized to spin up X number of resources without having to worry about security throwing a fit, or blowing an enormous amount of money.

But you must realise that the reason cloud costs are out of control is exactly because those budget meetings were actually not pointless.

>"There's less red tape to getting infrastructure provisioned"

Because companies do not like things going into assets category. Instead rent dedicated servers on Hetzner or wherever else it is price competitive and see red tape disappear.

>"So there's an incentive for developers to push to use a cloud so they can grow into their next role."

This is very lame argument. I am developer and a business and do understand both sides but sorry I run a business to make money, not to give them to Amazon and definitely not to be playground where developers can play with Amazon tech at owner's expense.

Because the claim is predicated on the wrong thing.

A company like uber is not looking at the price of a server but looking at the TCO of the products living in multiple verticals. TCO includes many things.

https://www.columbusglobal.com/hs-fs/hubfs/United%20States/B...

Reply on news.ycombinator.com