Hacker News Reader

Comment by Uptrenda | Hacker News Reader

Except that none of these compute resources are free and the cost of cloud-based super computing is a complete rip-off. It's actually thousands of dollars cheaper to build your own super computer from used rack mount servers than it is to use AWS, Azure, or Google cloud for lengthy computations.

As an example: you can buy a Dell PowerEdge M915 from ebay with 128 cores for ~$500 USD and a rack costs around the same. Five of them is 640 cores for a total cost of just 3k USD. That's 640 cores that you now own forever. Guess how much it would cost to use that many cores for a month on AWS? Well over 10k... and next month its another 10k... and so is next month...

With this option you only pay for power and still have the resources for all future experiments. I think the m1000e rack can fit 8 blades in total so you could upgrade to a max 1024 cores in the future! The downside with this particular rack is that it's very, very loud. But I've run the numbers here and it's hard to beat high-core, used rack servers on a $/Ghz basis.

View on HN

Are any compute resources free?

It's telling that you've neglected your own labor in this assessment, and that of the other specialists required to get such ad-hoc compute working at scale (dynamic software networks, on-demand compute, provisioning/billing/monitoring apparatuses, etc...)

What plenty have realized in cloud-native approaches is that the TCO is pretty compelling, sometimes because its easier to distribute procurement/provisioning/billing to teams for their own resource needs, or just in sidestepping the inevitable bureaucracy when you try to centralize that decision-making.

Of course its different if you're talking hobby kit setups in our own homes, but if you're aiming at real scale for absurd parallelisms then that ship has sailed.

I'm in the midst of returning from the cloud to coloc.

It's looking like for about $50K in hardware I get x10 the ram/compute/storage/(internal) network I get from the cloud at around $10K/mo. However, it is taking me about $25K in labor to setup. Hosting costs about $1K for a rack and a decent pipe to the server.

So more than a hobby but not approaching real scale.

The main barrier to using a lambda model is if you have anything that smacks of a DB. It would take me $100K+ to transition away from SQL. If you can do lambda w/o a big retool cost, then it is probably viable. If you are just running a bunch of VMs on the cloud, it is pretty expensive.

...

Interestingly, my main client (a very large company) just went the other direction and moved all its compute to a system that is Spark underneath running on Azure. They are trying to decommission some expensive TerraData instances. So far, it is a mixed bag -- it is a big step forward (for them) on anything that is 'batch-oriented' but 'interactive' performance is dismal.

Price appears to be a big motivation. I always forget that most large enterprises run on exotic stuff with crazy service contracts that makes the cloud look cheap.

> Price appears to be a big motivation. I always forget that most large enterprises run on exotic stuff with crazy service contracts that makes the cloud look cheap.

Don't forget the second factor of (not having) the workforce. Physical servers require a bunch of suckers who know physical infrastructure and accept to be oncall 24/7 and deal with DELL/HP/colo full time on top of periodic travels to the datacenter.

As opposed to cloud ops suckers who accept 24/7 on call and just sit around panicking when cloud providers are down and the status page is full of lies?

Colo providers have massive outages too so it's the exact same thing in that regard.

If we're talking regular maintenance, like a raid controller or a power supply going bonker. AWS is always accessible, it allows you to realize something is broken and create a new volume or instance in 5 minutes. Whereas with dedicated hardware you might be toast with no remote access and/or no spare parts.

There have been several AWS outages where EBS and EC2 instantiation was outright down, and you could not create new resources for a period of time. AWS is not “always accessible“ unless you’re in their marketing department.

Sure, you know you’re down, you simply can’t do anything about it. Not so with your own hardware, which is why many orgs continue to run their own gear.

> Sure, you know you’re down, you simply can’t do anything about it. Not so with your own hardware

I don't buy it. Everywhere I've worked that colo'd or owned their DC had wider-reaching outages (fewer than cloud, but affecting more systems). Usually to do with power delivery (turns out multi tiered backup power systems are hard to get right) or networking misconfiguration (crash carts and lights-out tools are great, but not if 400 systems need manual attention due to a horrible misconfiguration).

I think folks underestimate the non-core competencies of running a data center. Also often underestimated is the value of running in an environment designed to treat tenants as potential attackers; unlike AWS's fault isolation, when running your own gear it's really easy to accidentally make the whole system so interconnected as to be vulnerable--even if you make only good decisions while setting it up

Those "suckers" are easy to contract out to, and a lot of colo providers provide managed services or "remote hands" themselves as well.

In practice, when I was responsible for racks in two separate data centres, I spent 3-4 days a year in the data centres (combined); everything else was handled easily via tickets or remotely. Overall I've generally spent less time on devops with hardware in colo's than with cloud setups.

But specifically the amount of on-call work tends to be down to code quality and higher level architecture not whether you host in cloud or colo or rent managed servers - the "low level" problems become part of the noise floor very quickly in any system with reasonable failover.

Where are you going to store this thing? Pay for the power? Cool it? Replace failing hardware?

As someone who works for a company with over 150 data centers around the world, I know space and power is always one of our biggest expenses.

You can just store it at the datacenter.

Many of them offer a service which allows you to drop off your hardware and they will manage the power, cooling, even repairs and replacement for a monthly fee.

> You can just store it at the datacenter.

What do you think is ‘the’ data centre? There’s no single universal data centre. And most people don’t have a data centre, or access to one.

Actually everyone has access to managed data centre services.

It’s called co-location where you buy rack space in one of the core data centre hubs in each city. Been available for many years and it’s very affordable e.g. a few hundred a month.

Not sure why you said 'the data centre' rather than 'a data centre' though. Not sure if you think there's one massive data centre for the whole world?

I think GP said "the datacenter" rather than "a datacenter" in the same way they would say they have a computer at "the office" and not at "an office". Even though there's no "massive office for the whole world".

Aren't you a native english speaker? At "the X" is a common idiom.

E.g. people say "you can buy carrots at the grocery store" despite there being multiple such stores.

"At the datacentre" doesn't imply one universal datacentre. It just informs you of which kind of facility you can use.

Isn't 'the' a definite article. You use it when you mean a single definite instance of something. If you mean something in general you use an indefinite article, like 'a'.

https://en.wikipedia.org/wiki/Article_(grammar)#Definite_art...

https://en.wikipedia.org/wiki/Article_(grammar)#Indefinite_a...

"you can buy carrots at a grocery store"

"you can buy carrots at the grocery store over there"

From the same page you've linked:

The definite article can also be used in English to indicate a specific class among other classes: The cabbage white butterfly lays its eggs on members of the Brassica genus.

And from other sources:

We also use the definite article:

to say something about all the things referred to by a noun: The wolf is not really a dangerous animal. (= Wolves are not really dangerous animals.) The kangaroo is found only in Australia. (= Kangaroos are found only in Australia.) The heart pumps blood around the body. (= Hearts pump blood around bodies.)

We use the definite article in this way to talk about musical instruments:

Joe plays the piano really well. She is learning the guitar.

to refer to a system or service: How long does it take on the train? I heard it on the radio. You should tell the police.

It's pretty easy to get access to "a" datacenter. Just look up one that's around you and give them a call. I've done just that for the company I work for a few years ago. It's a pretty big DC provider, even AWS has a presence there. For a flat monthly fee, we get a full rack with cooling and power.

Almost everybody has access to a datacenter. There are countless datacenters that will host any machine you drop them, or you can rent their machines...

Surely you need to consider that monthly fee in your cost comparisons too then?

Sure, but that fee is very low from the options I have seen.

> Where are you going to store this thing? Pay for the power? Cool it? Replace failing hardware?

There are plenty of cloud providers who sell colocation services or even rent bare metal hardware.

Take for instance Hetzner. They rent 32-core AMD EPYC boxes wth 128GB of RAM for about 130€/month. If you rent a half a dozen of those boxes you get more computational bang for a fraction of the buck.

If you reach a scale where your needs span tens of data centers across the world then the economies of scale and operational expenditure are quite different and peculiar, but still they a couple of orders of magnitude cheaper than AWS.

You are right. And if you buy metal and rent some space in a DC, it is still more cost-effective.

You need a few people who understand a little bit of hardware and IPMI and you are set. That's not a big deal.

The cloud gaslighting is nuts.

Tons of companies now exist to give you insight in your cloud bill because that's the last thing cloud provides want you to have.

So in the end you must decide what kind of pain you want. Cost pain, tech pain, and that all depends on your company's particular circumstances.

None of this math is correct. You absolutely cannot get 128 cores for $500. Please link to a single valid-looking eBay offer at that amount (you may have seen 128GB and thought it was 128 cores). I see 32 cores for $500, at best.

Second, 640 cores on an m4.large for 3 years upfront (the equivalent purchase) is $8755/mo, not well over 10k.

Third, you really underestimate electricity costs if you're running a server 24/7 at all cores.

Last, and most importantly, most people simply don't run servers 24/7. They run batch computations for five hours a week, or a day, and then spend a week or two crunching the numbers.

There's some case to make that for certain institutions with very particular compute needs, running on-premise might make short-term financial sense (let's not even get started on labor costs). But it's really inaccurate to call cloud computing a "rip-off" for anything but the niche-est customers.

> Third, you really underestimate electricity costs if you're running a server 24/7 at all cores.

not to mention the cooling, or the per square foot cost of renting the space for this hardware. anything close to what's being described here is going to need some sound deadening (be it distance or material) from humans.

Frankly, I think many people here are overestimating the cost of electricity and cooling. My company leases several 42U cabinets with 208V 30A circuits in a modern SOC 2 Type 2 audited datacenter. We pay $600 per month per cabinet. Excludes IP transit costs. Granted this is in the Pacific Northwest where we have plentiful cheap hydro power.

> We pay $600 per month per cabinet.

Still a world of difference between AWS's 10k/month cost to access that many cores.

Comparing a Dell r740 (56 cores, 192GB ram, 1.6 kw psu), you need 11 of these for 640 cores (for a comparison with [0]). They're roughly 2500 each, so your up front hardware costs are 27,500.

Assuming the $600 covers power, rack costs and cooling, your data center bill is 6600/month.

For three years, the AWS bill is $315k, and the self hosted option is $265,100 (best case). I'm not up to speed with the pricing trends to say whether the gap will change over those 3 years based on data center changes or aws changes, but the difference is ~50k for three years, or roughly 3 months of an engineers salary over the cost of three months.

[0] https://news.ycombinator.com/item?id=23514698

Did you consider that AWS counts threads, not cores?

No, I did napkin math that compared an unverified number farther up the thread to the first server I found on Google. If you're considering buying hardware you should do your own numbers, I was merely pointing out that difference isn't 10k/month vs 27k up front

> so your up front hardware costs are 27,500.

That's about 3 months worth of AWS to access a similar amount of computational resources.

And if you don’t need them all the time , instead of 3 years RI you can us spot fleet or turn off half of the servers at non peak hour.

It's $600 per cabinet, not $600 per server. If you go by the PSU, that's 3 cabinets. But that 1.6kW PSU is quite overspecced for multiple reasons, like efficiency and the fact that it derates to half on 110v power. 28 core xeons have a TDP in the 165-255 watt range, after all. So even 1kW per server is overkill and that's only 2 cabinets. $1200 per month, not $6600.

With your numbers, that puts the total running cost at 71k, or a saving of $245k over 3 years, which is pretty much the salary of one person to look after that system for the lifetime. I'm not arguing that AWS is good value, im arguing that the upfront cost of purchasing the hardware isn't comparable to reserving EC2 instances up front.

You still need someone to look after your systems if you're using AWS. That's not something special to buying your own server.

And management of the hardware itself doesn't take an extra employee. It probably averages to less than one workday per month.

> With your numbers, that puts the total running cost at 71k, or a saving of $245k over 3 years

To compare apples to apples, that hypothetical pricetag buys you the system's max capacity for the entire period. In AWS, the invoice from using any of their serverless offerings grows wildly even when utilization stays way below the resources of that hardware bundle.

For instance, let's keep in mind that AWS charges:

* API Gateway per million of requests,

* Lambdas per timememory used by a single call, including the time it wasted with hot and cold starts,

SQS per queued message,

* WAF per request through the firewall,

* Etc.

> which is pretty much the salary of one person to look after that system for the lifetime.

My average time spend in data centers when handling racks in two different location was around 3-4 days a year. Buying on-call, out-of-hours support is going to cost you a tiny fraction of a persons salary per year, either on retainer and/or based on hefty day rates.

Hardware failures simply do not occur often enough for a rack or two worth of hardware to typically require enough maintenance to make it that expensive, and remote hands (people who act based on a ticket) are available in pretty much any data centre at low costs. If you get servers with IPMI etc. you typically pop in to wire up or retire equipment and dealing with the occasional failure.

Source: Have provided those kind of services for years.

On a side note, you would expect people to be super excited to explore possibilities of new infrastructure models. Instead, most posts related to lambda, etc... always come with this sort of reply.

For my use case (infrequent, easily separable jobs) FaaS stuff is a godsend. In the last year I've been doing things that would be orders of magnitude harder and more expensive thanks to it.

> I see 32 cores for $500, at best. Second, 640 cores on an m4.large for 3 years upfront (the equivalent purchase) is $8755/mo,

You can beat that cores per dollar handedly that if you're willing to use sufficiently odd hardware.

E.g. my Intel cluster here is mostly I have systems with 4x E7-8880v4 (22c@2.2GHz). I paid $150/chip + $450 host (thanks ebay) excluding ram.

(I have 8 such hosts now, so 704 cores)

So essentially one months cost for your AWS price w/ a 3year commit pays for an equivalent core count in systems for me... Throw in another month for the ram. Round up to another another to count for the (quite) non-trivial power usage.

You do not have to be anywhere near 100% utilization for AWS to be an extremely poor value large computing jobs compared to thrifty spending on the surplus market.

how?

>I paid $150/chip + $450 host (thanks ebay) excluding ram.

E7-8880v4 are $600 used each. Quad core mobos are not cheap, ranked ram is not cheap. Even if you pulled these from a vendor going out of business the bit coin mining crews would suck these up in heartbeat.

Where are you getting your prices?

Ebay. Hunt harder (and best offer harder). :) E7 CPUs sell absurdly cheap at times because it can be a bit hard to find compatible systems.

(similarly, quad socket systems with slow or no cpus go cheap from people that don't know much about them)

> ranked ram is not cheap.

Yes, ram for quad socket hosts it adds up, since you need at least 2-8 dimms per socket for full memory bandwidth (which was why I explicitly pointed out that I left it out).

On the plus side, those systems tend to have a lot of dimm sockets so you can use lower capacity dimms.

> bit coin mining crews

nah, not useful for that.

The way I look at this is you still need 3 people minimum on call to have around the clock ops for this rack. Three IT salaries is more than $10k a month. Eventually you'll find the balancing point but its higher than you think.

Yes. This is the reason cloud is popular. You don’t have to pay for computer janitors.

We have a title and don’t carry mops, my dear condescending developer who ostensibly doesn’t respect the entire ecosystem of people holding the Internet, the public cloud, and your entire CI/CD pipeline for whatever bullshit you’re cooking in NPM together. Do you have any idea how many “janitors,” most basically volunteer and most not employed by Amazon, are involved in the pedestrian aspect of merely making amazon.com land at the right place?

Yes, this is a good future: who would ever want to touch the computers they operate? Better to rent computers and position the core competency of your business with a third party, right? Because you can shed some of those pesky janitorial salaries? I’ll be waiting by my phone when you’re on the verge of bankruptcy once the clouds have their teeth in your books, and suddenly I get promoted from “janitor” to “computer operator who has my interests in mind, even though I not-so-subtly malign his existence”.

All you’ve done with this mindset is drive the same janitors to work for the clouds instead and contributed to the downfall of computing as a discipline that any player has any semblance of agency within, as the people who have actually touched datacenter equipment all work for them now or sit around in horror watching as a generation sympathetic to FLOSS arguments willingly hands over the reins of owning a computer to massive corporations.

Reading the comments in this thread already shows how much of 'those' 'software janitors'[1] don't understand what is required (and how little) to run on metal.

It's not that cloud is not the right answer. But people have started to forget that running your own metal is still an option. Or with current prices and performance: even a more viable option as it was in the past, because you can do so much more with so much less.

[1]: cranking out useless features nobody ask for while looking down on those (dev)ops people.

> software janitors don't understand what is required (and how little) to run on metal

Quite. In the middle of lockdown a client needed to spin up some virtual machine instances to demo a product to a potential client. Previous boss had been pushing a cloud-only strategy using Azure and was itching to retire all the physical servers.

Problem: can't spin up any kind of VM due to lack of Azure capacity.

me: "Well, we have that [physical] dev server which we still have, we could spin up the demo stuff on that..." new boss: "Oh, cool. Great lateral thinking!" me: <wtf?>

Demo done. Potential client happy. Boss happy.

Thanks for sharing this example. I got a call years ago from azure that I could not get the 10 vms extra capacity at that time. Insane.

Nice attitude.

Or you contract one of the vast number of consultancies that will do that for you at a fraction of that price.

And you still need someone available for a cloud hosted system - Over ~25 years of handling this kind of work, hardware failures have made up a vanishingly small proportion of the outages I've had to deal with, be it a colo'd setup or a cloud provider

You already have 3 people in on-call rotation anyway, regardless of the cloud or metal.

> you still need 3 people minimum on call to have around the clock ops

Q: Don't you still need at least 3 people on call for around the clock ops, regardless of how and where your servers or instances live?

> you can buy a Dell PowerEdge M915 from ebay with 128 cores for ~$500 USD

Where? Was not able to find one. Found a 32 core for $750. It was a pre-Epyc AMD server which has it's own problems.

There were only 9 options listed for the exact term you gave 'Dell PowerEdge M915' for the USA before I got to pricey international sellers. The few options make me feel skeptical about this.

This is of course before I even consider security and performance issues of used Intel servers.

Every time someone says on the Internet "You can buy x thing in Ebay for Y price" multiply the price by 3.

You forget a few things:

1. AWS and other cloud vendors don't have just one data centre per region, they usually have three redundant locations for availability reasons connected via a direct fibre link. 2. Cost of time; time to find hardware, prep hardware, maintain hardware 3. Cost of your agility. If you need more compute capacity, it will take you weeks to get the hardware required and installed, otherwise you need to have unused hardware sitting around "just in case". 4. Cost of your availability. What if you have a sudden spike of traffic within minutes and the current available hardware cannot service it? At least in the cloud you can spin up short lived resources to manage that load before it throttles back down again. 5. Permanent running costs of fixed hardware. A lot of implementations do not need permanent hardware running and can throttle down to a base of almost nothing so the average running cost on a monthly basis is actually very low.

You can't really compare value just by looking at core count, I imagine an old 128 core server off of eBay is going to perform much differently from something running on a more modern architecture.

However AWS lambda stuff runs on a pile of abstraction layers that have an impact on performance, not to mention their need to do cold starts.

You also have to account for burstiness. If you have nightly/weekly workflows, you might see some wild swings in resource usage, where an average usage and max usage differ by a lot. It’s nice to pay for average needs rather than max needs (especially when you can pay for on demand pricing).

>Except that none of these compute resources are free and the cost of cloud-based super computing is a complete rip-off. It's actually thousands of dollars cheaper to build your own super computer

Only if your time is worthless

" It's actually thousands of dollars cheaper to build your own super computer from used rack mount servers than it is to use AWS, Azure, or Google cloud for lengthy computations."

The issue is 'total cost of ownership' - not 'unit cost of equipment'.

The operational cost and overhead of running your own gear can be prohibitive.

AWS is successful because it is in fact much cheaper than the alternative in many enterprise situations.

AWS is sucsessfull becouse there is no capital tied to hardware for the leasers. For some reason it is what executives prefer. A 1000USD server is in practice impossible for me to order at work but I can use multiple 1000s a year in leased servers...

"AWS is sucsessfull becouse there is no capital tied to hardware for the leasers. For some reason it is what executives prefer."

Your execs are probably right.

Capital outlays are expensive and risky and imply a structural lock-in - but that's just the tip of the iceberg.

"A 1000USD server is in practice impossible for me to order at work"

And what about hosting? Networking gear? Support? Repairs and upgrades? Networking security? And how will those servers integrate with the rest of your outlay?

AWS is ridiculously cheap compared to the alternatives in most scenarios and that's why it's so successful.

AWS/Cloud enables so much more, far more dynamically - the value is considerable.

In some situations, if you have a fairly big need for computing, and it's predictable over long time-frames, and those services don't need to be tightly integrated with other cloud services, and you have the internal know-how to keep them running, the cost obviously savings can be achieved, but this is an optimization.

Put on your Eng/Ops/Business Hat for a minute and consider why services like AWS are exploding and growing to be one of the biggest segments in tech? Because 'stupid executives'? No - it's because the value add is fairly immense.

> AWS is ridiculously cheap compared to the alternatives in most scenarios and that's why it's so successful.

It's so successful because a lot of engineers selecting what to rent never see the invoices and don't understand the costs of colo or managed servers. You see it all over this discussion e.g. with people assuming a rack or two requires full time staff when you can get sufficient on-call resources on retainer for a few hundred a month or so depending on complexity of the setup most places.

Having ordered, configured, set up, hired staff for and run both colo setups, managed servers, and AWS setups many times I've yet to see AWS be remotely competitive on price ever, to the point that when I did contracting I used to offer clients to transition their systems to managed hosting (whether managed or colo comes out cheapest depends on scale and location - e.g. I'm in London and real-estate prices here are too high for colo to typically beat managed servers if you can deal with ~8ms round-trip to providers in Germany or France; for others managed is necessary to be close to customers) for a percentage of their first 3-6 months of reduced cost. That was costs including their devops contracts and monitoring etc.

I use AWS. It has lots of great features, and sometimes those features are worth the cost, but it's the expensive luxury option of hosting.

> Put on your Eng/Ops/Business Hat for a minute and consider why services like AWS are exploding and growing to be one of the biggest segments in tech? Because 'stupid executives'? No - it's because the value add is fairly immense.

I've done this for 25 years, including private cloud setups from before AWS was a thing, and I've had to yell at execs that wanted to triple our monthly costs because AWS sales had whispered buzzwords into their ears. That was the all in costs. It took a massive effort to explain it to them even with the numbers in black and white in front of their faces.

So, yeah, a lot of the time (not always), it is "stupid" executives. Sometimes talked into it by engineers working around planning processes that makes paying by use an easy end-run around budgeting.

The biggest achievement of AWS is to sell the idea that it is cheap because Amazon.

Triple the cost of on-prem is pretty good. Whenever I’ve run numbers for storage it has been in the 10-50x range.

This is for cost optimized on-prem storage vs ebs or s3. I check the actual workload and space utilization, and give amazon every benefit of the doubt to get the ratio that low.

It probably doesn’t help that a $100 Samsung EVO (500GB) can do 500K IOPS, but that the storage would cost $62.50 per month on a provisioned IOPS EBS volume. At a five year amortization, thats 37.5 times more expensive than buying. (EBS has poor durability by design, so you end up keeping the same number of copies on prem or in EBS.)

At that point, it’s basically game over, so it doesn’t matter that the IOPS would cost $32,500 per month.

Note: I always include the cost of the scale out infrastructure, etc in the comparisons, but with that, on prem is competitive even if the cluster is 10% full on average and does vintage stuff like triplicating data instead of erasure coding.

Not the person you’re replying to, but here’s another way of thinking of it: A $1000 server can sit under my desk, no problem. The TCO comes down to the cost of my productivity with the AWS bill as a second order effect.

If the ops team can’t manage >>10 machines for the price of me managing one, there’s something horribly wrong.

They should just let the developer buy the $1000 machine if it will help productivity.

That doesn’t mean production should run on a pile of machines under someone’s desk. That’s a different scenario.

Reply on news.ycombinator.com