Hacker News Reader

You can buy a 16-32 core machine with 64GB Ram for less than a hundred bucks on Hetzner. You can run SmartOS, proxmox, danubecloud or whatever other solution there, and for some of them you should be able to use terraform.

There is a little extra upfront cost to set these things up, but once you have it, it's fine. You can do your zfs snapshots to hetzners backup infrastructure.

A lot of people try to throw all their stuff on kubernetes, for that you can buy a few machines and provision them with ansible.

It's not like AWS devops engineers are the cheapest of all engineers, so there are a couple of solutions, which are mostly cheaper than their cloud alternatives.

Contrary to the parent I actually don't recommend going that route, mostly because I want to deliver a project and then have them be able to manage it on their own. E.g. an ECS cluster is simple to manage and maintain IMHO. I do tell them about the options that they have and them decide though.

I also haven't done baremetal server management, but my first concern with your first line is what do you do when as your services expand as does your dev team and the org needs more processes like log management, data warehousing, analytics, or whatever else AWS offers?

I'll admit this isn't always true (looking at you, Neptune) but I find AWS' key strength is the huge community, so most of its services have 1) decent documentation and 2) many users to refer to with technical questions as well as filling out roles.

buy as in own? Or rent monthly?

I do most of my work in large enterprises and there is definitely a point when you hit a complexity cliff managing your own systems. No one system causes this (major systems can often support a dedicated staff), but it looks like death by a thousand cuts when the HR interview system goes down after 7 years and no one is certain how to deal with its VMs/hardware/networking.

I have to say, you're completely overlooking why AWS/GCP/Azure are good services in your answer. The value add from using these services is that you don't have to know how to run any of these things, building an SRE/devops/sysadmin organization is non trivial for a team of developers with a background in application development. Spending an extra 100k a year on AWS is likely cheaper than the cost of building out a team to do all of these things for you, including as things scale.

If you're going to have a team of people deploying all of your own middleware software anyways on VMs, that completely defeats the cost savings from using cloud these days. It's much more of a PaaS business than a pure IaaS one.

I eagerly await CaaS (Cloud as a Service) so VPC, EC2 and EFS planning and deployment is automated.

AWS has a few pain points in the cost department; especially Traffic is insanely expensive compared to other providers. Hetzner costs about 1$ per TB of Traffic. EC2 costs 88$ for that amount of outgoing data, Cloudfront 80$, etc. That's not even accounting that Hetzner doesn't bill incoming and internal traffic, which AWS does.

There is no way that the little work I need to put up a network on a Hetzner host is worth 80$ the Terabyte of Traffic.

I have a regular Traffic usage of between 800GB and 1.4TB each month, AWS would easily double my monthly bill on that single item alone.

When you need a lot of CPU and RAM, AWS starts to get very expensive too, I have 128GB of RAM with 12 vCPUs and 2TB storage for about 55$/m, AWS has no comparative offering at even the remotely same price. That is even including the hours I have to spend sysadmin'ing this specific box.

This is great, how about adding reliability, availability, redundancy, SLA's and a few things more. How about needing to have a data warehouse, load balancers, SOC compliance? Not all the businesses go for the cheapest option and these decisions are rarely single dimensional. AWS is an integrated solutions for every aspects of an enterprise IT needs. If you try to compare a tiny slice of it with a single dimension comparison you can beat AWS but if you get the full picture it is getting harder to a point when you can't really beat it.

Availability over this year has been about 99.9% for the network, if I need more I can book a server in a different DC and do failovers. There is an SLA you can optionally book.

I don't require a data warehouse, load balancers (beyond HAProxy on the same host) or SOC Compliance at the moment, if I need those, I can build them.

Not all businesses go for the cheapest option but on the flipside if Amazon costs 80x of other providers, the business will probably just find two sysadmins and contract them for the work.

Even if you need bigger, load balancers are available as hardware, they're fairly cheap compared to AWS offerings and come with in-built firewall. Incoming traffic remains free on these. Your ISP will probably peer much cheaper than AWS and Hetzner if you're bringing enough money to justify it.

I know several corporations that do their entire IT out of AWS, some of them use inhouse and others scatter their usage across several cloud providers, AWS would likely increase their operating cost by 100x.

I guess you just downplay the amount of money you spend on developing your own solutions. What does a load balancer do on the same host anyway?

Where is this 80x-100x coming from? It would be impossible to create a 100 times cheaper solution because it would mean there is no margin left for the hosting provider.

Loadbalancers are useful for more than balancing load, you can distribute traffic to the different endpoints on them. They're fairly good at it.

80x is on traffic costs alone, I've detailed that above; 80$ for 1TB of traffic on AWS, not including incoming traffic costs, 1$ for 1TB of traffic on Hetzner, incoming traffic is free.

A 16core/64GB instance (m5.4xlarge) on AWS costs 360$ monthly, a tiny bit cheaper if you buy upfront. I pay 50$ a month for 16core/128GB/2TB. The 2TB storage would have to be paid extra on AWS. The m5a.4xlarge is a tad cheaper at 320$ monthly cost, again not counting for storage and bandwidth costs. I get double the RAM and lots of storage for less than 30% of the cost.

So on traffic, it's 80x cheaper, the instance is barely 30% the cost of AWS, and that's not counting the storage costs, which are very high on AWS compared to other providers (OVH, B2). And that only gets cheaper once you buy volume.

Of course the number 80-100x doesn't apply to me personally since I'm running a fairly low-scale operation but this all starts to stack up once you go large. A colo is even cheaper than any of these options since you only pay for power used and for hardware once.

To be fair, you should probably account for one or two mirror servers on hetzner for easier, lower latency fail over in case of hw failure (assuming you're talking about dedicated servers, not hetzner cloud). Of course with eg three servers, you might load balance acccross most of those during normal load, just make sure to have enough capacity left over to run with N less servers while you spin up a replacement and/or a disk is replaced etc).

Just for a more apples to apples comparison.

I guess the reason ppl don't "see" the insane premium clouds place on bandwidth is that bandwidth scales up with (presumably paying) customers. So as long as you're not streaming 4k video... You're happy to let the cloud eat out of the bit of your profits that "scales up".

We are heavily invested in AWS. AWS is not cheaper, but it doesn't mean it's prohibitively expensive when you consider everything it offers. What people tend to ignore are the other things it offers. For example, the parent talks about data transfer costs, but that's just one aspect of cost.

The real big cost in any organization is head count. And while a load balancer is not difficult to setup and maintain the first time, managing it becomes time consuming in a large enough organization. Couple that with everything else...

If someone can replicate what AWS is doing at a lower cost, people would move to them. But there are few companies out there that come close. Bandwidth cost is generally not your biggest expense.

This is a very interesting dynamic to me. Amazon seems to compete on price with EVERYONE on everything else, yet when it comes to AWS they’re so much more expensive than competition? Is this because their infrastructure is so much more expensive to run at 0 margins?

AWS still offers a very integrated experience and lots of people will simply pick AWS because of the brand name. Most people hosting SaaS or similar on the web don't actually need that much resources so for 98% of startups, I'd bet that AWS isn't the largest entry on their monthly running costs.

Their competition is mostly "old" guys who can easily and "cheaply" create and manage their own infrastructure.

In other words, when they "educate" young people about "The Cloud" and how is "The Best Way, the Only Way", they win because after a few years people got used to AWS as a fact of life and young professionals don't know how to administer a Linux box anymore.

Compare Amazon's (at least network) pricing to Google Cloud or Azure and its roughly competitive.

The scale of the networks these companies are building is unlike any of the novice-level stuff companies like Hetzner deploy. As one example, in one region AWS has deployed 5 Pbps of inter-AZ fiber. They build all of their own SDN switches/routers, all the way down to the chips now, for maximum performance and configuration with the AWS API. And don't forget, they maintain a GLOBAL private network; companies like Hetzner or DigitalOcean just peer to the internet to connect their "regions".

I'll keep saying this on HN until people start listening: If you buy AWS then complain about price, you might as well go buy a Porsche then complain about the price as well. They're the best. Being the best is expensive.

GCP is a real contender for potentially less money. It’s better than AWS in some if not many verticals. So I wouldn’t say AWS is “the best” (definitely not in the UI and user-friendliness department)

GCP is not nearly as good from my personal experience when it comes to support and reliability of services at scale. There are certainly some instances where GCP is easier to use, or they offer certain niche services that aren't exactly the same with AWS. Additionally given how close the costs are to begin with, it would very likely depend on your workloads as to which service was cheaper.

Overall I'd say Azure is the only real competition to AWS as a product (where they are often the most willing to negotiate a big deal), but googles open source efforts potentially pose a risk.

But do you need AWS' custom "GLOBAL" private network? I bet most would be happy to setup a cheap network tunnel between DCs if needed. There is plenty of existing tools to do all that for free (other than sysadmin work and I bet in a lot of cases even then the bill goes in favor of that).

I'm not buying a porsche and complaining about the price, I'm buying a car and complain it requires me to drive on roads made by the manufacturer and none of the parts are properly standardized and some things are expensive for no good reason.

Not disagreeing with your points, but internal traffic is free between the same AZ on AWS

Ye but I wouldn't know any projects that on AWS would need internal traffic that wouldn't also need instances over AZs/regions.

That's not even accounting that Hetzner doesn't bill incoming and internal traffic, which AWS does.

This is incorrect. AWS does not charge for incoming traffic nor does it charge for internal traffic within the same zone.

In an old project I had 2.8TB of transfer per month between DCs (ie zones in AWS). The hoster provided that service for free, AWS bills this for 20$.

Incoming traffic is charged on a few AWS applications, not EC2, but some do.

That doesn't really change the point though; AWS Networking is magnitudes more expensive than competitors and they bill for things that are accepted as part of the service in other places.

For large datacenters, excepting labor, on premises costs are usually an order or two of magnitude lower than Amazon, so you can ignore all the details. Specifically on labor, computers don't need that much of upkeep (unless you depend on flaky software, but then Amazon won't help you anyway), so it's a matter of having enough of them to justify hiring 2 people. if you do, you are better keeping them on premises. Besides, adapting your software for Amazon isn't free either.

For small datacenters, there are plenty of VPS, server rentals and colos around with prices an order of magnitude lower than Amazon. The labor cost of setting those up is comparable to the cost of adapting your software for Amazon, with the added benefit that you aren't locked in.

AWS has a few unique value propositions. If you require stuff very different from the usual (eg. a lot of disk with little CPU and RAM) it will let you personalize your system like no other place. If you have a very low mean resource usage with rare incredibly large peaks, the scaling will become interesting. But it is simply not a good choice for 90% of the folks out there. To go through your list, fault tolerance through Amazon is not that great, it need a lot of set-up (adapting your software), and adds enough complexity that I doubt it increases reliability; I don't know about compilance; about security, I only see problems; and integration is what AWS systems do worse, since network traffic is so expensive.

> For large datacenters, excepting labor, on premises costs are usually an order or two of magnitude lower than Amazon, so you can ignore all the details.

That’s too big a caveat to breeze by without a huge justification. If you look at the full cost that 1-2 orders of magnitude for many places goes negative and is a smaller margin for almost everyone else unless you have some combination of very high usage to amortize your staff costs over and/or the ability to change the problem (e.g. beating S3 with fewer copies or using cold storage, having simpler or slower ops or security, etc.).

It can be done but I’ve seen far more people achieve it through incomplete comparisons than actual measured TCO, especially when you consider the first outage where they’re down for a week longer because they learned something important about how data centers can fail.

If you have a simple business or website, AWS is not cost efficient compared to an old school shared hosting and it's not even more convenient. Those have slightly better control panels than the AWS Console and for crying out loud, AWS doesn't even offer automatic backups for their VMs as part of their offering.

They do now. See Snapshot Lifecycle Policy.