Twitter Search is Now 3x Faster | Hacker News Reader

Twitter Search is Now 3x Faster

engineering.twitter.com

17 pointsdenysonique17 comments

There are two ways to interpret this: OPTION A) Omg, I'm not going to use Rails, even Twitter is ditching it. OPTION B) Hmm.. They built something really fast using Rails, got it out there, got massive popularity, made boatloads of $$ and now they're maturing and moving to a more robust infrastructure.

I go with Option B. If you're a new startup -- don't confuse this advice and start brewing your own Java-based server just because Twitter is doing this now.

Boatloads of $???

AKA: Investors giving them money for worthless pieces of paper.

I think that's the way to look at it. You have to remember that Twitter is at a completely different level of traffic than your company will likely ever hit, even if it becomes hugely and wildly popular.

It's like looking at Google's server farm and saying "I can't be successful unless I too have a server farm like Google's."

The implication of Option B is that Rails isn't robust. A lot of people are going to avoid Rails because of that.

That could very well be true in some abstract sense; but who among us honestly needs the robustness and/or scalability of Twitter, really?

This article is not about Twitter ditching Rails, because Twitter is not ditching Rails.

This seems worth repeating [1] because this is the top-voted comment, even though the submission title no longer implies that.

Absolutely go with Rails [2], because it's a great way to get started quickly and large successful startups like Twitter are still using it at scale.

[1] http://news.ycombinator.com/item?id=2424393 [2] Or Django, or whatever

To clarify, Twitter hasn't and doesn't make boatloads of money.

Submission title is misleading. The main Twitter website is still running Rails and the article gives no indication that that will change. The article is about the Twitter Search frontend moving away from an architecture where their application servers would synchronously query the search servers, which wasn't scaling with their load.

Actually you are right, I haven't read it carefully enough.

Anyone from Twitter who can comment on the cultural/engineering ramifications of this?

I would be tremendously interested to hear from former Ruby hackers who have been shifting (back?) to Java via the Netty tool mentioned in the OP.

Definitely a different tack that we are used to hearing hear on HN . . .

Flagged.

Ignore the gauche of sending me a link to your service instead of the story for a second--consider your audience! Why would a HN user who clicks on a link to a story want to read your summary first?

If your system looks like it did the first day you launched, you're doing it wrong. It's difficult to communicate to this...

Am I the only one who reads it is only removing RoR from the search stack? It seems the rest of the front-end will probably exist as RoR in some form.

Maybe I'm reading between the lines here, but the feel of the article was "We got this code base from Summize and it worked great when we got it but now it's full of rot so we scraped it."

Twitter is big on the JVM and they have a bunch of Scala and Java engineers over there so this seems like the natural path. Besides, a lot of the big searching and map reduce frameworks out there are written in Java. This seems like a very natural transition.

The annoying thing is you're going to find a bunch of people who skim the article and start banging the "Rails can't scale" drum louder than they were 3 years ago.

there's not alot in here about dropping Rails particularly.... it's more some Rails powered front end stuff for SEARCH that needed to be ditched to speed up the crazy insane search load they get.

This is article is only talking about search.twitter.com, isn't it?

Expect them to drop RoR completely from Twitter soon. As far as I know they only use it for some front-end stuff anyway.

Reply on news.ycombinator.com