Hacker News Reader

Comment by amne | Hacker News Reader

This is all good advice .. for your hobby project. But the moment that server goes down and you have stakeholders draining your phone's battery you'll realize that scaling also brings availability as a very nice bonus. Sure, 99% of the time it looks like you're wasting resources. Until you need it and you do the math and realize you're way ahead in revenue not lost.

View on HN

My experience over the decades is an insider dirty secret: downtime doesn’t really affect revenue.

I haven’t ever worked for a business where this was true, including AWS. I’m sure they exist, but I think your experience is an outlier

AWS has plenty of failed data retrievals, outages not mentioned or documented, and other shenanigans of the sort. It's a turn and burn operation tho where skilled folk only spend a few quarters or years before moving onto other (usually more) stable tech companies.

You can hear all about this if you turn up to Harry's at 7pm on Thursdays, or at many of the other tech events in Seattle. Lots of current and ex-Amazonians out there with stories!

I’ve worked at AWS, working with high hundreds of customers during my time there. I haven’t ever seen someone using a service with irrecoverable data loss or an inability to use a backup.

Regardless, that wasn’t my point — the outages you mention do drive customer churn and cause sales problems. Ie AWS outages do impact revenue.

Ah, seems you missed out on the funnest shenanigans inside AWS :D

These are called Fight Club issues, and are restricted access issues that don't show up for most team members. Gotta cordon off the real hairy bugs!

I've seen a aws database segfault. Most likely due to custom aws patches.

I was referencing startups by and large. If you have 100 engineers or less.

I expect AWS outages affect revenue. But on the score of the number of companies… very few are in a similar position of AWS.

I used to think this way until in noticed that every clustering technology I had ever used had caused more outages than it had prevented.

Yeah, it often seems like in trying to eliminate the main single point of failure we end up with many more single points of failure.

Start by building a business that isn't differentiated by how many 9s you have. Something customers want so badly that a few hours of inconvenient downtime doesn't move the needle at all.

In this situation, blowing up your system complexity to maybe get another 9 makes no sense. Then the revenue change is pretty irrelevant for modest downtime.

People under estimate single server uptime. If availability is really that important, buy a hot backup. Put it in another region. Done.

AWS has major availability issues. VPS is not worse off here. If anything it is easier to debug.

Reply on news.ycombinator.com