Hacker News Reader

Comment by torginus | Hacker News Reader

Yup, I hope to god we are moving past the age of 'everything's fast if you have enough machines' and 'money is not real' era of software development.

I remember the point in my career when I moved from a cranky old .NET company, where we handled millions of users from a single cabinent's worth of beefy servers, to a cloud based shop where we used every cloud buzzword tech under the sun (but mainly everything was containerized node microservices).

I shudder thinking back to the eldritch horrors I saw on the cloud billing side, and the funny thing is, we were constantly fighting performance problems.

View on HN

Tangential point but why is it that so often these leaving the cloud posts use the word "beefy" to describe the servers? It's always you don't need cloud because beefy servers handle pretty much any bla bla bla

https://hn.algolia.com/?dateRange=all&page=0&prefix=true&que...

If anyone from oxide computer or similar is reading, maybe you should rebrand to BEEFY server inc...

>Tangential point... rebrand to BEEFY server

idea for an ad campaign:

"mmm, beefy!" https://www.youtube.com/watch?v=j6ImwKMRq98&t=21s

i don't know how "worldwide" is the distribution of Chef "Boyardee" canned spaghetti (which today is not too good), but the founder's story is fairly interesting. He was a real Italian immigrant chef named Ettore Boiardi and he gets a lot of credit for originally evangelizing and popularizing "spaghetti" and "spaghetti sauce" in America when most people had never tried it.

https://en.wikipedia.org/wiki/Ettore_Boiardi

you know, "spaghetti code..."?

The servers are always beefy and the software is always blazingly fast. Blazingly beefy is my new joke trademark.

Because that is a casual word in the English language to describe an object with substantial power?

If you would suggest a word that would make a better substitute in this case, that could move the conversation forward, and perhaps you could improve the aesthetic quality of posts about leaving the cloud.

Well, I wouldn't say I have a beefy car or a beefy sword: there's some historical bit of linguistic connection that seems to have caused people to describe a server with the adjective "beefy", rather than "powerful", "hefty", "stacked", or... "chonky" ;P. Coming up with some options, I think my favorite might be "cranked"?

The first definition for beefy from Merriam-Webster is: heavily and powerfully built .

As someone who's had to rack some pretty heavy servers with lots of GPUs, CPUs, NVMe drives, and RAM, that turned out to be quite powerful, I'd say "beefy" is accurate.

https://www.merriam-webster.com/dictionary/beefy

Side note edit: hefty and chonky don't tell me much about power. Stacked is usually more about a team or group of things (and for individuals, I'd rather not see it).

I wouldn’t describe a corvette as beefy but I might call its engine beefy and i would definitely call a diesel truck beefy.

I think servers, especially bare metal, are in the category of grunt and raw power. Beefy feels right.

Because the server types you get for the price of a single Heroku dyno are incredibly beefy. And suddenly you need a lot less dynos. Which is quite important if you start managing them yourself.

Noted!

My conspiracy theory is that "cloud scaling" was entirely driven by people who grew up watching sites get slash dotted and thought it was the absolute most important thing in the world that you can quickly scale up to infinity billion requests/second.

No, cloud adoption was driven by teams having to wait 2 years for capex for their hardware purchase and then getting a quarter of what they asked for. You couldn't get things, people hoarded servers they pretended to be using because when they did need something they couldn't get it. Management just wouldn't approve budgets so you were stuck using too little hardware.

On the cloud it takes five seconds to get a new machine I can ssh into and I don't have to ask anyone for the budget.

You can save a lot of money with scaling, you have to actually do that though and very few places do.

And now, on cloud it’s the same but much more expensive and worse performance. We’ve been struggling for over a month to get a single (1) non-beefy non-GPU VM allocated on Azure, since they’ve been having insane capacity issues, to the extent that even “provisioned” capacity cannot be fulfilled ;-(

Sure, but that’s because Azure. I’m sorry someone made the decision to go there. AWS & GCP, stock outs at least used to be nearly unheard of.

Until you hit a certain scale.

I totally agree about Azure being the worst of the three, they wanted us to commit to certain use before even buying hardware themselves. Crazy…

But I also had capacity issues with Google at large scales in many zones.

What sort of scale, if you don’t mind me asking?

Hey, sure! That’s important context.

One gameserver was 40vCPU and 256G of RAM, we had about 30-50 before we’d see some issues in some regions. (this is from memory now unfortunately).

Sao Paulo and Tokyo being the worst, but Singapore, Australia and Mumbai also had issues at various times.

The other places where we hit hard limits was Los Angeles, but we had more than a hundred instances then.

The issue with the hard limits is that it’ll be one zone thats exceeded and the API will fail, so you have to retry with another zone in the same region, but you don’t get to practice building your autoscaler before you actually need it.

Oh interesting. Yeah, and I’m guessing you don’t get much prior visibility into available stock, since that would also expose info to their competitors.

That is a lot lower than I expected, but I also imagine that’s a sizable order that they like getting.

If you're a top X customer running in a smaller region of AWS or GCP, yes, you need to do capacity planning with your TAM. You get to a point where quota increase requests are not auto-approved.

I think this is partially that for the past decade or two, on-prem was partially preferred by very frugal companies.

One of the places I worked that was on-prem enforced a "standard hardware profile" where the servers were all nearly the same except things that could be changed in house (like RAM sticks). When they ordered hardware, they'd order like 5% or 10% more than we thought we'd need to keep an "emergency pool".

If you ended up needing more hardware than you thought and could justify why you needed it right now, they'd dip into that pool to give you more hardware on a pretty rapid schedule.

It cost slightly more, but was invaluable. Need double the hardware for a week to do a migration? No problem. Product more popular than you thought? No problem.

>I think this is partially that for the past decade or two, on-prem was partially preferred by very frugal companies.

Sure this is made worse by frugality, but I experienced this problem when virtualization was in its infancy, much less cloud anything even existing much less being popular.

There's "cloud" as "server in the cloud I can use" which is what the majority of smaller players are using - it's just someone else's server.

There's also "cloud" as the API-driven world of managed services that drain your wallet faster than you can blink.

This. Instant availability of compute and storage, purchasable with a corporate credit card, was the cloud killer app.

I can't have instant ressources, azure Switzerland is overbooked and can't create much ressources, at two different clients. And I read something similar for AWS.

And in most "large" companies, you still need to go to different teams and processes to have those cloud ressources.

Oh and if your company made the mistake to outsource their IT, odds that you'll have a 4 digit bills to change a teraform.

Welcome to Western Europe.

Seems like a few negotiation skills would be of better use than doing extreme amounts of work so somebody can take months on approving new hardware. Well guess what, the people that did slow down hardware procurement are now slowing down deployment of cloud resources as well because the fundamental problem wasn't addressed and that's organizational misalignment and disfunction.

You're going to negotiate with the structure of an entire corporation?

Excuse me CEO your budgeting process is inconvenient to my work, please change it all to be more responsive.

This is not how things work and not how changes are made. Unless you get into the C-suite or at least become a direct report of someone who is nobody cares and you're told to just deal with it. You can't negotiate because you're four levels of management away from being invited to the table.

An organization that can make agile capital expenditures in response to the needs of an individual contributor is either barely out of the founder's garage or a magical fairyland unicorn.

You would be surprised. I have unclogged processes inside and between companies before where people would tell me it's not possible or that success is unlikely.

And yes, if you are working with professionals most expenditures can be planned well in advance so you definitely can take months or even more than a year in the process. If there is a major issue affecting the business of the company you would be surprised what is possible. I've got things approved in weeks in an enterprise of 2000+ employees before and I was 5 levels down from the CEO then. I have improved my negotiation skills greatly since then (it helps when you co-found a consulting company).

Even if you can get instant approval, Dell or HP won't Fedex you 50 servers next day like they will 50 laptops.

And once you're a customer you get to deal with sales channels full of quotes and useless meetings and specialists. You can't just order from the website.

You can get servers elsewhere as well and they can be more agile than Dell/ HPE/ SuperMicro/ Lenovo. Asus, Asrock, Gigabyte and some other brands can ship servers rather quickly. Perhaps not the next day but the prices are much lower out of the door especially if you are a smaller customer. Most companies know well in advance how much compute/ storage they will need. So if something takes a month or two it isn't a big issue.

> On the cloud it takes five seconds to get a new machine I can ssh into and I don't have to ask anyone for the budget.

This isn't exactly realistic, not for too long anyway.

Once your cloud bill climbs into the millions, expect to see just as much scrutiny on what's costing so much and what can be cut and can you really justify the new thing you want to spin up (as there should be).

Having been through many growing startups, I'd say the freewheeling days of spin up whatever you want only last to about 50K to 100K/month AWS billings.

>to a cloud based shop where we used every cloud buzzword tech under the sun

this is me currently, not so much "tech" as repackaged cloud services as SAAS and middleware, department is amortizing hours and effort against projects, the eldritch horrors you're referring to are starting to manifest and I want out.

However old, .NET Framework was still using a decent JIT compiler with a statically typed language, powerful GC and a fully multi-threaded environment :)

Of course Node could not compete, and the cost had to be paid for each thinly sliced microservice carrying heavy runtime alongside it.

I don't think we need to bring languages and frameworks into this. Some of them make things worse, others -- much better.

Furthermore, the microservices craze only made things worse regardless of PL / framework.

IMO we have an entire generation (maybe two) of devs who never self-hosted. That's the main audience of the article and of many of the comments here.

The rest of us who only scoffed at the cloud and microservices were just waiting the world to start coming back to its senses again.

Reply on news.ycombinator.com