Comment by nirvana | Hacker News Reader

Comment by nirvana | Hacker News Reader

I'm sure Amazon will fix this issue, but the question it brings to my mind is- why are so many people using ECC? Most startups need hosting, not an elastic compute cloud. ECC makes sense for someone who needs to spin up 1,000 workers, pull data out of S3, process it, store the results in S3 and then spin down the workers when it is done.

But startups need hosts that are up 24/7. ECC doesn't give you any guarantee of uptime, and if it goes down the local (fast) disk is ephemeral. Yet, the EBS alternative which is backed up, is very slow.

The basic VPS offerings, that Linode, Rackspace, and just about everyone else offers, isn't available at all from Amazon (near as I can see.) Yet this is what startups need- local disks, a small monthly fee, and up all the time.

So, Amazon requires extra engineering--to account for nodes going down more often and to make ephemeral disks reliable or EBS performant. It also puts you on the path to lock-in, since so many of amazons services have their own unique APIs. It isn't exactly cheap, near as I can tell, when compared to, say, dedicated hosting in germany.

Its advantages exist elsewhere- if you need to spin up a bunch of machines with an API you can do this at rackspace cloud. And... that's about the only advantage I'm able to think of. Personally, I'm architecting to have some overcapacity built in and to survive a spike, because I use that excess capacity in the off hours for heavy lifting. I wouldn't try to bring up extra nodes in the morning and shut them down in the evening anyway... and I doubt that many startups are really doing that.

Possibly, I'm missing something. I tend to forget about features that aren't compelling to me, but are compelling to others. So maybe there's something that's important to these startups.

Having used EC2 for web hosting I can affirm that you are exactly right. Web hosting on EC2 is a challenge and quite expensive. From my experience EC2 is much better suited for large data storage, distributed computing, and any other use case that can tolerate EC2 quirks, hiccups, and outages.

S3 is a big part of it. If you're storing lots of data, a "local disk" just isn't enough. Now you're spending your time solving problems Amazon already solved.

Occasional downtime just isn't that big of an issue for many startups, who are still looking for product-market fit.

I see what you're saying, and I failed to illuminate that I'm coming from the perspective of someone who is working with a cluster of riak nodes. Riak is a ring topology much inspired by the Amazon S3 design. I'd never trust anything to just one spinning rust device, sure. I was taking a distributed storage architecture (using whatever open source platform you prefer) for granted. But I recognize that S3 predates many of them.

If you expect to grow your start-up quickly, it makes sense to use a provider that allows you to scale. It is very costly to maintain two separate infrastructures for different providers.

I think I'm misunderstanding your point. Why would you need two different infrastructures? Anything you can host on ECC you can host on backspace cloud, but without being ephemeral. Or you can order servers by the dozen from other providers.

Maybe you're talking about ordering servers by the hundreds, but are successful startups like foursquare doing that? Seems a few a week is what they'd add, and you could do that with most other hosts, without the added complexity of Amazon.

The difficulty of scaling is not in getting the hardware, but in the operational burden and frequency of failure. With EBS, machine and disk failures are effectively irrelevant and you can instantly migrate to another machine. You don't need to worry much about data loss. If a disk breaks in one of your machines, EBS keeps working. If the machine breaks, the data is still in EBS. If EBS breaks, the back-up is still in S3.

Dedicated hosting and rackspace cloud may be cheaper, but if you expect to be in a situation where failure becomes frequent, you'd better be on EC2.

With AWS, the development, operational and financial burdens are higher, and the frequency of failure is much higher.

EBS is not a unique feature, as there are multiple equivalents of EBS, along with higher level methods of replicating data, namely, couchDB, riak, cassandra, etc.

But I think you've answered my question- people use AWS because they don't realize this.

If you are starting up, where are you going to get the capital, expertise, and people to operate and maintain those systems? You have better things to do.

I love it when people tell me spending more money and time and effort on something that is less flexible, reliable or performant is somehow saving money, time and effort. I think you do not understand what you are talking about and you're giving me marketing spin, in fact that must be the case. You're simply wrong. I've explained it, you're not listening.

Of course the kicker is that this is in a thread about how a bug in Amazon means that the EBS volumes that were allegedly being backed up, actually weren't.

As tybris's comment indicates, the allure is primarily that you can configure the system to automatically spin up new instances on-the-fly as load increases, in case your startup gets on the front page of reddit.

Because most startups want to gratify that vanity, they set up on EC2 because they want to prepare for the "inevitable" deluge of users without paying for an adequate pipe or server configuration around-the-clock.

Reply on news.ycombinator.com