Hacker News Reader

Comment by zxcvbn4038 | Hacker News Reader

I’ve found that are generally three stages in a company’s life. If you are extremely technical you can make bare metal beat cloud in the beginning. As the environment grows and IT roles diversify cloud starts to make more sense financially then bare metal so you do a cloud migration. If you are lucky then eventually you get to the point that it’s cheaper to run your own hardware then rent someone else’s and you end up migrating out of the cloud again.

If there is a fourth stage then it’s getting to the point that it’s more profitable to run a side business renting out your unused resources to others and you become a cloud.

I’m surprised there isn’t a Walmart cloud or an Exxon cloud by now.

View on HN

> If there is a fourth stage then it’s getting to the point that it’s more profitable to run a side business renting out your unused resources to others and you become a cloud.

This is a common belief about AWS, but I find the counterpoint persuasive - if AWS involved Amazon renting out its unused resources, it would have to shut down in December. But obviously AWS does not shut down in December. They're not renting out their unused resources; they're renting out their experience running this kind of system.

This is our exact journey so far.

1. On prem hosting managed by 1 person who works out of broom closet in some budget colo. Zero compliance or official procedures involved.

2. Cloud hosting managed by whoever needs a thing taken care of. (We are here)

3. On prem managed by dedicated in-house team in multi-colo setup. (We really really want to be here)

I think we will skip the 4th stage. If we are forced to build our own datacenters, something went very badly.

The 30k/m figure posted elsewhere in this thread is a fairly reasonable line in the sand for jumping from 2 to 3. We also have unique regulatory requirements that could move that threshold lower.

3. On prem managed by dedicated in-house team in multi-colo setup. (We really really want to be here)

another major advantage of this kind of setup is that it is possible to give far better (technical support) to your customers, especially if you provide infrastructure services to your customers.

having a competent technical team which can do proper indepth analysis of issues customers are having has a massive positive impact on customer relations. Compared to most technical escalations i have seen at most of the cloud providers, which comes down too "it was an internal problem, but is is fixed now, don't know if this will happen again".

> it is possible to give far better (technical support) to your customers, especially if you provide infrastructure services to your customers.

This is exactly why we want to do it. Being able to offer our customers a 100% vertical solution stack that we completely control & understand is a massive selling point in our industry (and hopefully others).

oh there is definitely a market for this, and cusotmers willing to pay for it.

at $job, me and another team member basically did technical deep dives into issues that occured for $customer. Usually they needed deep root cause analysis as of why something happened. (Mainly for insurance/legal/compensation reasons).

It was a ton of fun and we even got to a point where customers where willing to pay for this service when required. It costs us a lot of hours to do the troubleshooting, and not all customers have such requirements.

Some fun root causes i have found over the year:

- a bug in a firewall which resulted in kernel crash of the packet processing plane (and thus packet loss). We worked extensively with the vendor to fix this. This bug only occured because $company was using a custom TCP stack, and was sending ACK's incorrectly every X packets.

- tons of issues regarding time. Time drift on routers, time drift between processes inside a router. Time drift between hosts.(this happens a lot, and it makes me appriacate the fact that some routers have external clocks as a source option).

- quite a lof of issues with IPsec tunnels. For being a standard, i have never seen suchs a lot of edge cases with non-interoperability.

The fourth stage doesn't even involve owning your own infrastructure.

We pay a 3rd party for our AWS (and Azure) usage to get better rates than AWS would give us (and without a commitment to increased spend like AWS reps have tried to push for Amazon's available programs).

I wouldn't be surprised if they just used volume discounts / negotiated pricing and reservations at their end to make it massively profitable for them.

I think SPOT (pre-emptable) instances are an under appreciated resource in AWS. Technically they can get pulled back at any time but in practice it’s only happened to me once, for a single instance. I’ve lost far more instances to hardware failures in the same period. If you can get your head around that then you can save more money with SPOT plans then any of the regular compute plans. AWS also does an auction for SPOT instances which can push the price really low for ARM and previous generation instances whereas other clouds just give you a set discount when using them.

If you've only had a spot instance pre-empted from you once then you haven't used them very much. We have a very elastic workload that can scale up to a few hundred spot instances and it's not uncommon to see tens of them yanked out from under us at once.

Reply on news.ycombinator.com