Hacker News Reader

Comment by ergo98 | Hacker News Reader

Reading both this entry and the one that explained why they went with AWS, I'm left confused about why they ever went to AWS in the first place.

View on HN

Having built a non-trivial production system, I have a love-hate relationship with AWS. I love the possibilities and potential, but I'm deeply disappointed when it fails to live up to the hype.

It's just a matter of alternatives. There's frankly no other IaaS out there that can match the features of EC2 (eg, EBS, elastic IP). These guys are unstoppable, they come out with more features every week.

Yeah, EC2 is the best cloud but it's a shame that it's still missing features that ESX 2.0 had (and Amazon has shown no interest in fixing it).

Wes, what features are you looking for?

Not the original poster, but here's some stuff I really wish AWS had:

a) Ability to map an elastic IP to an ELB. The CNAME thing it uses now has way too many drawbacks [1].

b) Ability to make an RDS instance a slave to or a master for a normal MySQL instance. This would make it possible to use RDS as a backup to our ordinary DB infrastructure using normal MySQL replication (and eventually, vice versa)

c) Retention periods for EBS snapshots. You can get this yourself by writing a few simple scripts, but it would be really nice if snapshots could simply be labelled "delete after 90 days".

d) A cross-availability zone, synchronously replicated EBS volume. This is probably fairly specific to our use case, but it would be neat if AWS natively provided something like DRBD.

[1] http://blog.pagerduty.com/2010/08/31/load-balancers-need-sta...

It's not me in particular, but if you read the EC2 forum going back to 2006 people have been asking for full virtualization, multiple IPs, multicast, non-NAT, reverse ARP, shared EBS, etc.

Disk I/O performance that doesn't suck.

Let us add dev pay instances to an ELB.

More ram.

Elastic private ip addresses.

Change security groups of running instances.

I have a lot more, but those are my big ones.

While it is true that you cannot add new ones or remove existing ones, you can modify an existing security group of a running instance: http://aws.amazon.com/articles/1145?_encoding=UTF8&jiveR....

Do you work with AWS?

Me too. This feels like "oooh, shiny!" from high levels, not engineering needs. You shouldn't outsource your core competency, and this feels like what they are doing.

I say "feel" advisedly. I don't have inner knowledge. It's just that the explanation up to this point doesn't seem all that compelling; for what seems like rather dubious benefits it seems like they've taken on an awful lot of risks they have little ability to manage themselves.

This came up in the first entry. Netflix would apparently dispute your take on their core competency.

http://techblog.netflix.com/2010/12/four-reasons-we-choose-a...

The problems [Amazon] are trying to solve are incredibly difficult ones, but they aren’t specific to our business. Every successful internet company has to figure out great storage solutions, hardware failover, networking infrastructure, etc.

Their third argument for virtualized infrastructure was "We're not very good at predicting customer growth or device engagement", which strikes me as a compelling problem to solve. Having to change their approach to managing service dependencies and failures doesn't seem like too high a price to pay.

Yes, I read the first entry. The use case that particularly came to mind when I said it seemed like they were outsourcing their core competency was actually the location of data centers and other such physical constraints, like peering arrangements. They're stuck with Amazon's decisions on that front. Granted they're probably large enough to get Amazon to do whatever it is they need to do, heck they can probably all but simply purchase a new data center and announce to Amazon that Amazon now has a new data center to manage :), but it still seems like an odd choice of control to release.

the location of data centers and other such physical constraints, like peering arrangements

This is not Netflix's core competency either. This is the core competency of a CDN. If you have problems like this, you should almost certainly hire one, and architect accordingly.

Netflix has long since done so:

http://blog.streamingmedia.com/the_business_of_online_vi/201...

Actually Netflix now uses Level3's (level3.com) CDN: http://www.level3.com/index.cfm?pageID=491&PR=958

Notice how they say - "primary content delivery network", primary not sole.

netflix actually uses many cdns, including akamai and l3.

No, they still use Akamai (dig cdn-0.nflximg.com to verify), but they are planning on moving to Level3 (assuming the Comcast/Level3 fight doesn't sour that deal too badly).

I think they alluded to it in the first post. When they rolled out Xbox 360 streaming, they severely underestimated demand. It was nearly impossible to get a full quality stream for 2-3 months after the roll out.

Now with AWS, they can spin up additional instances at will.

Note that Netflix does not use AWS for streaming.

Can you please provide a reference?

Sounds like the initial investment into reengineering their software to handle distributed environments well turned out to be quite large, which is why it does sound a bit like they brought the troubles upon themselves. But presumably they now have 2 advantages: ability to scale horizontally, and having their software adapted to distributed environments (which means that technically they are not limited to AWS, in case problems arise with Amazon's service). Sounds like the foundation of their house is stronger now, which was probably the whole point of the exercise.

In short: Because restaurants don't farm their own crops or raise their own livestock. They cook.

Analogies merely confuse (they always have that superficial "aha!" moment, yet never bear scrutiny). If it must be, however, in this case it's more synonymous with a restaurant that farmed out their kitchen and their wait staff, but they carefully crafted the menu, chose the silverware, and did all of the marketing.

That is an interesting business proposal.

It seems to be the franchise model: you buy the land and hire the grunts, we'll give you the Big Book O' Brand and handle the marketing.

Reply on news.ycombinator.com