How much does The Cloud really cost?

I'll add that it isn't just data egress. It's also inter-AZ transfer which costs $0.01/GB. But it's really $0.02/GB since you get charged 1 cent for the traffic leaving and 1 cent for the traffic arriving on the other side.

If you're running in AWS, you want to have servers in multiple availability zones in the same region so that if one AZ goes down you're still up. Let's say you're running your APIs and Kafka in two AZs (us-east-1a, us-east-1b). A user hits your API in 1a and you publish a message to Kafka. You need to write to 3 boxes. On average, you're paying $0.03/GB for your Kafka writes - $0.02/GB when the API server is in the same AZ as two of the quorum and $0.04/GB when the API server is in the same AZ as only one of the quorum.

That's 30x what Hetzner is charging for internet egress from its US data centers. People here were outraged that Hetzner would charge $0.001/GB for internet egress while using AWS is costing 30x that for transfer in the same region. Plus, a lot of companies use 3 AZs (so that they still have high availability if one AZ goes down). Then you're paying the $0.04/GB on all writes.

And this doesn't just apply to Kafka. Let's say you're running PostgreSQL with standbys: you're paying for all that data transfer from the API nodes in other AZs to the leader node and all that data transfer to replicate the writes to the standby nodes. When reading, you could do AZ-aware reads from the standby nodes if you don't need the latest data to avoid charges.

At some point, it feels like we're spending a lot of time and effort trying to minimize cloud costs - costs that we don't have to worry about from traditional datacenter providers. The cloud was supposed to make things easier, but the costs have made a lot of stuff harder.

And I'm not saying that AWS should charge nothing for inter-AZ traffic - they're just charging way too much. I don't know how much it really costs to service that traffic, but Hetzner can send the traffic over the internet for $0.001/GB from data centers that they don't own. It feels like Amazon should be able to send that traffic down the road for a good bit less than that. I believe in charging customers for what they're using so that you don't have bad actors exploiting pricing loopholes - but Amazon charges an absolute fortune for stuff that's relatively cheap.

As someone who works for a company that’s transitioning out of the cloud into their own data centers, the supply chain factor is difficult. You have to be really good at forecasting and planning with a large upfront cost. But the savings are substantial (up to 70%)

That's always a possibility, but let's say you're running a cluster of 100+ machines. One dies and it'll be 24 hours to get a replacement. That's not such a big deal - your capacity is only down 1% and you aren't running stuff at 95%+ utilization.

If you're running 1 box, you might have a point. If you have 100 boxes, you could pay for a couple spare boxes and still save a ton of money. If you're paying $700/mo/server for AWS and bare metal would cost you $100/mo/server, you could have a few spare metal servers and still pay a ton less. 100 servers * $700/mo = $70,000. 103 servers * $100/mo = $10,300 - the "extra" $300 is kind of a rounding error.

Even if you're running 5 servers, $700 * 5 servers = $3,500; $100 * 10 servers = $1,000. You literally have a spare server for every box in your cluster. Heck, even if you're running 1 server, $700 * 1 server = $700; $100 * 3 servers = $300. You have double the number of backup servers already online for less than half the price. Instead of running PostgreSQL, having a server go down, and then restoring from a backup to a brand new cloud box, you could literally have 2 hot standby servers and still save money.

Yes, bare metal isn't instant like the cloud - but if they're charging you 7x more than bare metal, you could literally have 2 backup servers for every 1 server and still save a ton of money. You're not paying an extra 10% or 20% for the cloud's flexibility or ease of replacement. You're many several times more for every server - to the point where you could just purchase 3x or 5x with bare metal and just have some of it laying around idle.

Why do you only have _one_? Get two or three. :-)