Hacker News Reader

Comment by jeffwilcox | Hacker News Reader

Data centers aren't cheap, so unless you have the economies of scale to offer R&D investment and stock-based compensation to your employees to build a modern cloud DC, good luck with that... done right you can save operating expenses, but it'll take a huge investment that would not scale for others.

View on HN

I strongly disagree. Datacenters are super cheap compared to EC2. (I'm not talking building your own: start by leasing space from existing datacenters). There are a surprising number of places where you can go and lease a rack or ten or a whole room and be up and running in a couple of months.

I make the case that colocating pays off at just about any scale, assuming you have $10k in the bank, have a use for at least 40 cores and are able to pay upfront to handle anticipated scale.

Hurricane Electric has prices online of $300/mo for a rack. On AWS, a single full c4 machine (36 threads) costs $1.591 per Hour x 24 x 30 = 1145/mo -- this is more than the cost of running a whole rack with 40 machines. Decent internet can be gotten for hundreds per month.

Ok, so how about buying your own machines? E5-2630 with 20 threads is $700 x 2 = $1400 + motherboard + disk + ssd brings it to several thousand, so it will pay off in at most 6 months, and we're not even talking bandwidth or storage costs. Depending on the application you could be looking at a payoff after 2-3 months.

Worried about installing or remote management? IPMI, iDRAC, etc included with basically every server make this a piece of cake.

The only good case for cloud are if you may suddenly scale 10x and can't predict it; don't have $10k in the bank; or don't have 1-2 months to order machines and sign a contract for rack space.

What you're missing in that comparison is an extra engineer (or two) who can deal with power needs, networks hardware config, firmware updates, plans for rolling hardware, stock of replacement drives, seamless base system updates, dealing with the platform (virt or containers) itself and other things that the managed cloud gives you included in price. Ah, and they need to be available on call. Hardware may be cheap - people aren't.

You need people to manage AWS instances too, ive never seen any evidence that it actually takes less people at scale.

Sure for a few instances, its likely to be true because there is a certain amount of fixed overhead for dedicated hardware, but it remains a relatively low constant, but in reality there isnt much difference in terms of labour between wrangling hundreds of AWS instances, and hundreds of servers, and the servers will be many times cheaper to run.

This is a good comparison with EC2, but doesn't directly address the comparison with many other AWS products.

Nowadays, you don't need to do that yourself to run a cloud because you have companies like Joyent offering fully managed private clouds at a fraction of the cost of AWS. They will rack and stack, manage the cloud software, provide escape plans so you aren't locked into them as a vendor and provide architectural guidance to your app teams.

DISCLOSURE - I am a Joyent employee.

Even Facebook leverages colo facilities as some of their POP's. Still, I think it's common that somewhere near the mothership these big guys build a facility from the ground up.

FB uses colo facilities as POPs because that's where the eyeball networks connect. It's strictly a latency/connectivity play.

If the revenue is there, it’s a no brainer to build out your own data center or acquire prebuilt space. Cloud provider margins become your cost savings (clearly, as evidenced by this article). You’re going to need infrastructure people regardless, whether it’s on-prem expertise or AWS/Azure.

It’s disingenuous to inflate the compensation for datacenter employees as a boogie man, or to wave away a “modern cloud DC” as a Herculean undertaking. There rarely is any R&D required, and stock compensation isn’t always necessary.

Fair points, appreciate the insight. I think living in a tech bubble city clouds my thinking a bit much.

The only real example I have good knowledge of (1 of 1 cases) is a tech friend who worked for a firm acquired by a company wanting to build out data centers, and in the end everyone left after their acquisition stock awards vested, before really adding value to the corporation and delivering on the promise, because the company did not have a "tech r&d" comp plan in place to keep the subject matter experts employed. In the end the project and the DC expansion fell short.

I can appreciate that living in tech hot spots can tilt the understanding of certain domains. “Unknown unknowns” if you will.

Most people I know who build datacenters don’t expect a tech R&D budget or outrageous compensation, as it’s general contracting, racking/stacking equipment, and infrastructure engineering (pxe booting, remote consoles, reimaging for OS and/over hypervisor, etc).

Here’s a non-comprehensive list I generated on the fly yesterday of tech orgs who run their own datacenters or equipment in colos:

https://news.ycombinator.com/item?id=16449973

Reply on news.ycombinator.com