Hacker News Reader

Comment by ayb | Hacker News Reader

I use Heroku for subscription software services, online retail stores, and phone ordering system for our staff.

Right now all of our sites are failing with 503 errors. Our store is down and when one of our employees went to take a phone order they got a "Welcome to your new app" message.

I've been a big evangelist of Heroku since we migrated over last year, but I'm getting deeply concerned about the elevated error rate since every minute is costing us money.

View on HN

Does Heroku have an SLA? (I could not find it)

At some point they're exposing themselves to serious risk. Rackspace had to pay out ~$3MM (in free service credits) after an outage in 2009:

http://www.networkworld.com/news/2009/070609-rackspace-outag...

This is offtopic, but what's MM? What's the second M for?

M = 1000 in Roman numerals, but the confusing bit is not reading them like Roman numerals (2000) but rather interpreting them as one thousand thousands, or one million.

Hmm, so who uses this? How is it better than just saying $3M?

MM is the standard abbreviation for million in the financial world. "The great thing about standards is that there are so many to choose from."

This came up in a thread a while back (http://news.ycombinator.com/item?id=1483667). Bottom line is that "MM" comes from the banking/finance world. In banking, $3M actually means $3,000 and $3MM means $3,000,000.

I see, thank you. I prefer the SI, kilodollars, megadollars, etc.

My question to you is, could you do better and how much would it cost? If you didn't use Heroku or another cloud provider you would pay a lot more up front to get your applications running. When things go wrong you would have to fix it, which means paying technical staff to be on call. Since you and your company are likely experts in your domain and not in infrastructure then any infrastructure that you built would likely have more downtime than Heroku. You have to debit the cost of Heroku's downtime from the cost of building your own infrastructure.

Disclaimer: I'm not affiliated with Heroku and I don't use their service.

You have to debit the cost of Heroku's downtime from the cost of building your own infrastructure.

That's silly, and also not how it works, at all. You're paying PaaS/IaaS companies so that it's their headache, not yours. Once it becomes your headache, they are no longer doing their job, and you are no longer receiving value for which you are paying for. You don't debit the cost of their downtime from the cost i would've built your infrastructure, you debit the cost of their downtime from your business' revenue and reputation.

Whether or not you could do it better yourself does not excuse the downtime one bit.

Excusing or not excusing are irrelevant.

If you stop using Heroku and manage your own infrastructure, you need to take all relevant costs into account.

Of course, finding an alternative provider of the same (or similar) services is also an option.

Isn't it commonly recognized that it's cheaper to run your own hardware than to pay a cloud provider? It just requires more capital outlay and maintenance.

We're actively sending pay per click traffic to our online store and it's very easy to spend hundreds of dollars. When our traffic converts it's great but it pains me to think I could be sending traffic to a Heroku 503 error page and have zero control over it.

So, "could we do better"? I'm not sure. I'm trying to figure that out. It certainly would not be as easy to use as Heroku or easy to deploy. But at a minimum I need to get some other host option set that we can switch over to.

Is there a service that will switch off adwords campaigns if your site is down or in maintenance mode? If not, there should be.

Thought about writing an app to do that. Unfortunately I would not be able to host it on Heroku. :-)

I've had clients with sloppy dev teams who decided to change the URL structure of all landing pages without letting me know (I was managing their PPC campaigns). Google stops serving ads after getting 404 errors - unfortunately I don't think they count other errors (like a 503) and they don't stop until they've sent a few hundred (or thousand) clicks.

would ppl pay for a good solution here, like $10 a month, or a % of money saved? I imagine that with a good implementation they might.

If you run your own site and things go wrong, you (hopefully) know what you did. When Heroku (or AWS, or anyone) makes a change, they don't consult every customer to find out if now is a good time to go down.

Are there any companies that provide 'server host failure' insurance for instances like this? It seems like a possible opportunity.

The E&O insurance I looked into getting when I moved into consulting would have covered it -- "lost sales" resulting from a "hardware or software malfunction." I assume if you start making recurring claims on that the insurance company will reevaluate whether they want to continue doing business with you, though.

Don't know specifically about "server failure" insurance, but I assume it exists. There is insurance for practically everything. For example, an Uncle of my friend builds home security systems, and he is insured in case a home he has secured is ever broken into anyway.

I've currently got a few small apps on Heroku and am considering moving some larger ones over. But the "Heroku | Welcome to your new app!" is very worrying.

Custom error pages for these kinds of errors would be very useful.

It took Google App Engine two years to add the option to specify a custom error page for server errors and over quota errors. Hopefully that'll come soon for Heroku as well…

It will. Currently in private beta, heroku will render an iframe pointed to an arbitrary url hosted externally (say, on S3.)

Eggs, basket, etc.

Reply on news.ycombinator.com