Amazon Data Center: 450,000 Estimated EC2 Servers | Hacker News Reader

Amazon Data Center: 450,000 Estimated EC2 Servers

huanliu.wordpress.com

115 pointsbenatkin14 comments

In public lectures by AWS employees they have clearly described their infrastructure as 1U-based. Heat dissipation is a big part of their costs, and blade servers trade heat for space, which is a bad trade, as space is cheap compared to heat management. I'd assume that their infrastructure is not dense.

Blade servers also cost more than 1Us on capex, still.

In public lectures by AWS employees they have clearly described their infrastructure as 1U-based.

They've also described their infrastructure as consisting of racks of 20 servers + a switch, and shown photos of racks full of 2U servers. My guess is that they have multiple generations of hardware.

Yes, considering the size of their operation, the length of time they’ve been running, and the variety of instance types they offer, it would be really surprising to me if they didn’t have multiple generations.

Also, they don’t offer {4,8}XL cluster {compute,GPU} instances in Oregon. If their infrastructure were homogeneous, there would be no reason not to. (Well, other than that those instance types are probably likely to have less latency-sensitive applications.)

Clearly they run a pretty tight ship, but it doesn’t follow that something on the order of half a million machines all have exactly the same mount. And they could be 90% 1U-based (justifying the public statement) but, going by these figures, still have 45,000 non-1U servers.

Makes sense to run 2U servers; more room for independent disks; no RAID, no redundancy, 1 disk per virtual machine "slice", so you don't have contention issues when someone else on the box tries to do a dd from EBS to the local non-persistant disk.

1U pizza boxes wouldn't have enough physical drives to break up among the amount of virtual machines you can cram on a box with 64-128GB of RAM.

You honestly think you get a disk per instance? Or that a node needs a disk?

Based on non-persistant disk IO tests on an EC2 instance, yes, it appears you are getting a raw disk.

Power density will definitely be higher, but blades don't generate more heat per server, do they?

I don't think that they generate more energy emitted as heat, rather it is that heat dissapation becomes much more expensive at higher density.

More comments on that regard

http://news.ycombinator.com/item?id=3706753

This reminded me of the German tank problem. http://en.wikipedia.org/wiki/German_tank_problem

I'm wondering what the "revenue capacity" is for the the entire farm?

I don't know how they virtualize, but even a conservative calculation on instance capacity (eg. assume each server can support one XL instance on average) means that they can conceivably bring in $288k per hour.

I'm sure I'm way off with this estimate, is there any evidence out there are regarding their typical server configuration or utilization rates? Spot instance prices paint a pretty vague picture about utilization (eg. rates are pretty much flat over the last month).

I posted yesterday but fell soon out of scope: http://news.ycombinator.com/item?id=3703456

I think it would be interesting to know if they are planning to design and make their own servers like Facebook: http://www.theregister.co.uk/2011/04/07/facebook_data_center...

With 450,000 servers don't you think Amazon could do a better job of recommending books for me than just finding books just like the one I just read? I get better recommendations from the new books table at any self-respecting bookstore. It's about what I _haven't_ read!

Reply on news.ycombinator.com