How Facebook Copes with 300 Million Users

technologyreview.com

36 pointsmgcreed5 comments

A question that's been on my mind: Why did Facebook scale so well and Twitter scale so poorly? Is there really any way to learn how to cope with huge load before you actually hit the point where you need it?

From the article it sounds like Facebook made a huge investment in people to build the bespoke (software) infrastructure that they needed. eg. A bespoke in-memory database, their "own storage system called Haystack that's completely built on top of commodity hardware", etc.

Facebook have an order of magnitude more employees than Twitter which probably makes those kind of bespoke uber-scaling advancements feasible.

A bespoke in-memory database,

They use memcached with some nasty kernel hacks.

(Caused some friction when they forked an old version, added new code which was then suddenly unleashed on the community without warning. Each new feature then had to be manually added to the current version)

Scaling Twitter: Making Twitter 10000 Percent Faster http://74.125.113.132/search?q=cache:mfi_ssC0kxwJ:highscalab...

(highscalability.com is down. perhaps they need highavailability.com?)

Such an awful problem to have. I feel horrible for the employees there. =)

Reply on news.ycombinator.com