How to scale your rails app, by Scribd (a YC startup) | Hacker News Reader

How to scale your rails app, by Scribd (a YC startup)

scribd.com

26 pointsmichelson0113 comments

Presentation is nice, but I would like to suggest a feature for you guys. Instead of that terrible flashy frame, why don't you by default show documents in HTML? Browser handles scrolling, selecting and picking font size much better.

If I want to print or store PDF, I see "download as PDF" button already.

Also, what's with the popups? I have not seen them because my browser blocked them, but there appears to be some critical functionality that relies on popups being allowed.

Very nice. It definitely looks like the scribd guys have a clue.

Scribd is still still running on just one web server. That means they're not at the stage yet where they can just throw machines in to handle growth. That's a pretty significant step. Their one web server just happens to be so damn beefy that it can handle a lot of traffic. It will max out at some point though.

That means they're not at the stage yet where they can just throw machines in to handle growth.

Hmm, why do you say that? They've already got the DB off on other machines, and it sounds very much like they are doing all the things you do to a rails site so that you can scale it by plugging in more app servers. I.e. they already put everything shared into the DB or memcached.

Most people discover fun problems when they split their web servers, no matter how prepared they thought they were. I'm sure they'll handle it well, but it's a significant step.

It isn't really a significant step. You just plug in another web server and make sure the load balancer and DNS settings are right.

Yes in theory you're correct. In practice things are usually dirtier.

What if they don't have a load balancer? What if they want redundant load blancers? What if they use round-robin DNS and then need a failover procedure? What if round-robin DNS isn't balanced enough for them? What if logs files need to be merged? What if the web servers aren't 100% identical? Data needs to be sync'd? Load distributed more intelligently? Cron jobs modified? etc, etc.

Perhaps they kept things unusually clean and simple, but most sites require a little finagling when they go to multiple web servers. I'm not saying it's difficult, but I do think it's wise to expect a few bumps.

Hm... I don't understand why did they opt to go with TWO(!) SQL servers so early. If your DB is designed well and your queries are written well, you should max out on HTTP side first.

Even on super-fast ASP.NET, we started with two HTTP servers plus only one single SQL server behind and it worked very well...

Scribd is pretty write-heavy - between traffic analytics, hit counts, votes, comments, favorites, and submissions, they're looking at several DB writes per page view. Good indexing and DB design can make your reads really fast, but your writes are still limited by the speed at which the disk rotates. And if you have a lot of indexes to support fast queries, they'll slow down your writes by some (fairly small) percentage.

You're right though - most of the sites I've been involved with have been heavier on the webserver than the DB. It depends on the application...

According to the presentation deck, they have 3 DB servers. It's not clear to me what each of them do, but one of them apparently gets activity logging writes, and the other is a slave that they use to do analysis queries on that data.

I would like to know more about the overall architecture especially with their backend written in C# rather than Ruby.

this talk was given at the silicon valley rails meetup a couple days ago.

how did they just launch and already get 1 million hits a month?

They're one of the most popular sites on Digg - not the site itself, but individual pieces of content hosted on the site. It's the YouTube effect - if you build a site that's viral, you grow exponentially, but if you build a site that hosts viral content, your growth rate is exponential.

Reply on news.ycombinator.com