Comment by thelastgallon | Hacker News Reader

Comment by thelastgallon | Hacker News Reader

Parent Full thread

These are the features that AWS provides

(1) Massive expansion of budget (100 - 1000x) to support empire building. Instead of one minimum-wage sysadmin with 2 high-availability, maxed-out servers for 20K - 40K (and 4-hour response time from Dell/HPE), you can have 100M multi-cloud Kubernetes + Lambda + a mix-and-match of various locked-in cloud services (DB, etc.). And you can have a large army of SRE/DevOps. You get power and influence as a VP of Cloud this and that and 300 - 1000 people reporting to you.

(2) OpEx instead of CapEx

(3) All leaders are completely clueless about hiring the right people in tech. They hire their incompetent buddies who hire their cronies. Data centers can run at scale with 5-10 good people. However, they hire 3000 horrible, incompetent, and toxic people, and they build lots of paperwork, bureaucracy, and approvals around it. Before AWS, it was VMware's internal cloud that ran most companies. Getting bare metal or a VM will take months to years, and many, many meetings and escalations. With AWS, here is my credit card, pls gimme 2 Vms is the biggest feature.

> (2) OpEx instead of CapEx

Someone please explain to me why this matters. I'd think that expenditures are expenditures, and that if the outright purchase of hardware would see an RoI compared to renting it in the cloud in under a year, it'd be a no-brainer to just buy the hardware.

OpEx means that if demand for your service goes down, cost goes down, your hardware does not become a capital liability since it depreciate fast. Way easier to justify changes to it too, you don't need a purchase project to get new instances, you're already "approved" and the contract was already signed with fluctuating costs. Needs more hardware? press a button, no need to research vendors, get contract negotiations in place.

AWS makes the life of finance and leadership a lot easier because they spend a lot of money justifying their superiority in ways that you don't have to think too hard to use and be taken seriously. They're to CTOs what think tanks and lobbyist are for lawmakers.

"No one got fired for buying ibm" for the new era.

There is a lot of truth in AWS propaganda, they're great for many things. But some of it is built on lies, cost being one, performance another.

The problem with those 5 people, is you can't hire a 6th - your stack is custom and probably even if you find the guy, he'll need months of ramp-up.

In contrast, you could throw a stone into a bush and hit an AWS guy.

If your 6th needs months to understand how the basic blocks in your system are arranged then he might not be one of the "good" guys

Not really a hardcore infra guy, but on the coding side, I know companies with products that have codebases in the multi million LoC range written over decades, one of my friends interned there and told me they didn't even let him work on the core product for months, they put him on some custom testing framework they had for it, just so he could get familiar enough with the core code to be able to contribute meaningfully.

He told me that before they started doing that, there were incidents like teams writing entire modules they didn't know already existed - now there were 2 pieces of code doing basically the same thing, that were just incompatible enough to not be possible to merge them.

And how does AWS help with this?

On the infra side - by standardizing things.

One time, in on prem, we had a custom setup with a machine running half the services we used, including a reverse proxy using haproxy with some custom Lua scripts for routing, a fileserver using lighttpd, some docker compose stuff, a stateless query thingy running on nodejs, etc.

We needed to change something, and the guy who wrote it left a year ago and we had to reverse engineer the stuff he did (some of it was quite questionable).

We weren't entirely successful and had to rewrite some stuff. I'm not saying how it was done wasn't clever or cost efficient, but damn if it was done on AWS, I probably would've known where to look for stuff (and so would've most of my colleagues).

Bare-metal management isn’t custom, it’s just not cloud. A TFTP server, your choice of confirmation management, your choice of image builder, probably - but not necessarily - a hypervisor, and that’s pretty much it.

This stuff isn’t hard, and it was solved well over a decade ago. Probably two at this point. It’s just been shoved down by the hyperscalers (who themselves are busily hiring for those skills).

Why would you need more people? Don't treat the 5 people like shit.

You say cloud allows massive expansion like it's a negative but it can be boon for a pre-pmf startup or a scaleup. You simply don't have to worry much about capacity planning in cloud and that can be a huge time/effort saver.

Sure, if you're only growing <30% YoY and already paying several millions for cloud and bandwidth/storage are large fraction of that, by staying in cloud you're proving your incompetence as an engineering org.

You say as if every startup scales to facebook size within months. Most startups are hilariously small and their workloads won't stress out a mac mini. Of course, you could overengineer and spend too much and hire too many people, but thats on the people running the startup.

Great, put it on t4g.small or Render or something, what's your point?

> one minimum-wage sysadmin

The internet assures me there are loads of these underemployed Unix/networking experts just sitting around waiting to set up your infrastructure. But in my experience, these people are actually really difficult to hire, and not at all cheap. (Possibly the sharp ones have 'sold out' and gone the SRE route and are now one of those '3000' people.)

So I wonder if there's a certain amount of wishful thinking on both sides here, like "I wish a 'clueful' company would hire me to be their head sysadmin...", while companies who have tried to do this on the cheap usually just have terrible ops. ("Whoops, the backups haven't worked in 2 years...")

Yeah I'm one of them. Started as an on-site Linux sysadmin. Moved to cloud SRE because remote is plentiful and it pays better.

I get crap recruiters in my inbox and LinkedIn every other week with the worst offers to go back to on-site bare metal admin. 30% less pay, on-site requirements, and it's a contracted position?

I need that Futurama "oh you're serious, let me laugh harder" gif

If companies want to whine that good Linux datacenter ops doesn't exist anymore, laugh in their faces.

Reply on news.ycombinator.com