Comment by bogota | Hacker News Reader

Comment by bogota | Hacker News Reader

Parent Full thread

Unfortunately AWS is the new oracle. No one ever gets in trouble for picking it and its a great way to make it look like you as a high up exec provide value. Look how fast we are iterating now with my decision. It almost always ends in a mess of unmaintainable unthought out services that someone else has to come and clean up or move to the next proprietary service.

The last 5 years for me has been soul crushing as someone who actually enjoys managing datacenters. We have seen time and time again having your own DC leads to much better visibility and control on spending as well as lower cost. Not to mention the huge advantage when negotiating with cloud vendors if you are a mid size or up company.

So time and time again i have had to transition out of environments you can reason about into AWS and become a glorified support engineer but i guess thats what companies need now days. Someone who will read docs the other engineers dont want to and troubleshoot all the issues because AWS is so easy.

Im glad I got to learn how the “cloud” works though as i likely never would have been drawn to infra and programming in this day and age.

The problem with running your own DC is growing past your planned capacity. There's often a huge delay between developers having to put up with the VM infrastructure having to put up with under-resourced machines and more capacity being approved.

As a developer I've put up with over-subscribed VMware clouds and I vastly prefer the Azure/AWS option.

Their are a lot of bad ways to run your own hardware. Limiting your infra to EC2 for burst capacity provides an easy escape hatch if you need. However i have never run into the issue of not calculating CPU capacity properly and also would never use VM ware that sounds like IT is running your DC. I could see this being the case for some tiny startup who just owns a small amount of rack space though.

Of course you can pay the cloud providers to deal with your companies bad planning. Thats what most do.

I also would never advocate for DC for everything. As a startup it likely makes no sense to run your own hardware and also likely doesnt make sense to run k8s also however one of those is completely acceptable. Once you get to more predictable growth owning your own hardware starts to look more attractive but most don’t know how to calculate it properly and finance likes to make it merky with capex and opex buckets.

I recently experienced this firsthand, in a company which owned no computers beyond employee laptops. The product was entirely built of AWS services created by a pile of Terraform spaghetti. It was only really understood by someone whose superpower was the ability to keep an apparently unlimited number of levels of indirection in his head.

I hear they might need to move it all to Azure soon!

Terraform just does too much and is abused. Its great for simple config but quickly turns into custom modules to make things easier. Eventually those modules need to change and stuff breaks but you never know until someone trys to run it again. Then you just pray no one trys to manage their database with it.

Additionally as its put together over time if you actually had to re create the environment it would never work. I spent time automating recreating an environment and quickly stopped. Terraform is the illusion of infra as code and fails miserably at any scale.

You know you can lock modules versions, right? And of course the environment recreation needs to be tested periodically if you have a disaster recovery plan.

Locked modules only works if all of your modules are pulled in through the registry. that isn't an option at some places.

Testing environment recreation is impossible at a certain scale. Try it when you have hundreds of people adding terraform code most of who only know how to copy and paste.

> I hear they might need to move it all to Azure soon!

I'm not sure if you made that post as a joke or not...but you just described a certain San Francisco based startup that operates in the SPF/DKIM/DMARC space...

Which company (if you can say) were you describing?

Many people have been fired after AWS migrations resulted in massive cost increases. But, few people want to talk about failed projects at hospitals etc where the cloud is a poor fit.

IT has always had issues with people cargo cutting solutions without understanding the details, and the cloud is no different.

I agree its just the current one that affects me. We did have someone get fired from an AWS migration at my first startup but they just turned around and tried to do again. It took one year for a single application that only lived on a single server and we never did anything more advanced before we sold.

Reply on news.ycombinator.com