Comment by nethergoat | Hacker News Reader

Comment by nethergoat | Hacker News Reader

Parent Full thread

Designing for fault tolerance and scaling out instead of individual node durability and scaling up is, in most cases, a sounder approach. Since the vast majority of applications map well to this case, it makes sense for Amazon to focus on features that assume/assist scale-out.

It's easier to design for scale-out than one might think; a great resource to get started is http://highscalability.com.

Regarding EBS, I haven't seen the hangup issue you've described. Any data?

As for low inter-node latencies, Amazon has an offering that specifically addresses that need: http://aws.amazon.com/ec2/hpc-applications/

Overall, I think Amazon is making a lot of inroads in areas with specific hardware demands. They just launched GPU compute options, and I expect we'll even see SSDs soon.

Scaling out vs. scaling up is the false dilemma, the answer is both at the same time solving the bottlenecks you have using the time and resources you have. The "vast majority" of applications do not map well to an n-wide map-reduce-esque design, the "vast majority" of applications bottleneck on an authoritative datastore's I/O capacity.

I'm the sysadmin (not the dba) for a website that does 30K http req/s at the edge, and our mysql cluster does about 3000 req/s during mid-day with most queries in the 1 - 5ms range.

Highscalability is a terrible place to get started, its a great place to share notes but if you get started there you will waste insane amounts of time on architecture astronaut nonsense.

What do you consider architecture astronaut nonsense?

On the spectrum of scaling an application AWS has definitely leaned hard towards scaling out. (where the other end is scaling up) This means it caters a little more to denormalized databases and load balanced web servers than it does to one giant mysql database server or a cluster of a few large database servers.

I agree with nethergoat in that if you need low latency communication between ec2 instances that you should be using their hpc instances with faster/optomized ethernet.

I however disagree with nethergoat and agree with cagenut and think that the vast-majority of applications still currently depend more on a scale up paradigm. That is slowly moving towards scale out as people/businesses see its benefits and AWS is catering to this trend.

Reply on news.ycombinator.com