Why does digg need so many servers?

Everything that is cool about Reddit at this point is what Redditors are doing with Reddit, not what Reddit is doing with its site. And isn't that how social media is supposed to work? Not just some notion that users are tools, that you'll crowdsource your way to being yet-another-traditional-media source (except without journalists)... social media should be individuals forming their own communities... communities of interest, communities of practice, communities of support -- all of which exist in Reddit (/r/SuicideWatch both disturbs and impresses me).

So the fact Reddit-the-software isn't changing that much doesn't seem like that big an issue, it's more like infrastructure. A major redesign would be negatively disruptive to the communities that are building there. Not that there aren't great things Reddit could do but isn't, but I think they are doing well at the most important stuff. If Reddit had 68 employees they not only would be fat, they'd probably fuck up a good thing.

But that's the funny thing, is the Reddit is growing and Digg is shrinking- despite their staffing. Reddit does a much better job at allowing the community to manage themselves on a volunteer basis with the moderators/admins of the subreddits than Digg.

Absolutely. Every time someone trots out "And Reddit only has 7 guys", I think to myself that they're innovating as if they had 3. Say what you will about Digg, but at least they rolled out new software.

Where is Kevin advising people to take less money? If you mean the $80 million comment, that was almost certainly before the Series C, so digg had less than half the funding.

"Kind of makes you wonder what all the digg employees do all day."

Communication. I'm always surprised that so few people who do software (I'm talking management at companies) are familiar with the mythical man month. Adding employees to a software project always yields diminishing returns due to the increase in communication required.

Unless they are using the really big instances though, I find that EC2 nodes are generally lower in power compared to what most people are using for their servers.

I remember not too long ago before they moved to EC2 everything fit in just a few racks. Of course, their traffic has expanded significantly recently.

It depends how you approach the problem. I'm guessing here, from my own experience implementing social graph features, but here goes:

There are a couple of ways you can go about this, the first is the database: Join the network of people against the land of content and bring it back. This doesn't work (as an aside this is what people mean when they say web scale, it has nothing to do with web traffic, it's social graphs) your database will cry. All though not at first, in development it works fine, and you feel fine, and for a while you're ok, but you start growing....

Another way you can go about it is by denormalizing. In this world you store a pointer to each content item for each user. So anytime I do something all the people [following|watch|connected|friended] to me get a record indicating I did this. This works, but now you have lots of data (lots and lots of data!) spread all crazy around. You need some kind of system to push that data out to everybody. It's those last two that drive up your hardware usage, it's not necessarily web boxes, but it's boxes in the background broadcasting the events out to the world, and the datastores to hold it all. Depending on how your web code works you could also have a lot of overhead on the webservers putting all that stuff together.

My experience here comes from building the social features into toolbox.com. A good example is this page http://it.toolbox.com/people/george_krautzel/posts-connectio... That's all the posts from users connected to our CEO (all 750k of them). Getting that to return in near real time is super fun (and you can probably tell that I went down the DB join path before it all fell apart).

Both are simple sites by any definition. The database queries for almost all the pages views are clustered index seeks (on SQL server) and in MySQL they are mostly looking up on the primary keys for the tables used (these are MyISAM tables so no clustered indexes here).

Both are equally unoptimized outside of the database queries which I believe are fairly well optimized.

No real cacheble content on either outside of the usual suspects (JS, CSS, some images)

Not in a drag race where he's driving a Yugo and you're driving a Ford GT. Unless you crash of course.

I'm not sure what you are suggesting?

Free coffee would be enough to get me to say something nice about Microsoft products?

I work at Microsoft?

Facebook has a lot more than 100 times digg's registered users, though.