How We Built Airbnb Holiday Cards in Five Days | Hacker News Reader

How We Built Airbnb Holiday Cards in Five Days

nerds.airbnb.com

65 pointsAirbnbNerds13 comments

TL;DR: Monolithic applications (in any technical stack, not just Rails) are bad. See also: http://en.wikipedia.org/wiki/Big_ball_of_mud

Extricate focused concerns (like authentication or service discovery) so they can be well-tested, well-designed, and reused in new services.

I'm curious about how exactly to go about extricating authentication, but I'm not sure where to look. Any pointers for best practices or cool patterns for this?

We have a single sign-on (Rails) application along with a client library that other apps use to integrate with it (which provides helpers for checking for accounts and roles as well as rspec helpers for writing tests that involve logging in without retesting the login/signup flow)

Ah, ok. Do you think it would work okay with a distributed setup, with different apps on different servers, and so on? I'm imagining lots of RPC calls just for authorization. (Like, if the apps do defensive role checking.)

Initial discoverability is the one thing that most SOA articles I read tend to skip over or hand-wave away which really drives me nuts.

What else is there out there apart from ZooKeeper?

Doozer: https://github.com/ha/doozerd

ZooKeeper is really the most accepted option here. It is used by many other important projects (Kafka, Storm, Hadoop) so adding it to your stack is likely to pay dividends later when you want to adopt some other cool technology that relies on it.

Doozer is the other option I've heard of that fills this role, but I don't personally know of anyone using it.

In terms of initial discoverability, services register in a designated place in ZooKeeper using an ephemeral node and then other services can find them there. You can set a "watch" on the ephemeral node to get notified if it goes away for any reason (ie. the other service or node goes down)

Really interesting read. I've only ever built one feature websites so I never came across this problem, but it indeed is an awesome problem to have. "Lots of users, lots of features, what do we do?" must be such a fun problem to solve, especially when everyone is getting paid. I need a job :(.

Have you tried using Rails engine for this or different components of your app?

It is actually pretty neat because the same login session can still be used within multiple engines. You can also release them as gems too.

I used to work on an enterprise app which we created the whole chat system with Rails engine. Then, we release the engine as a gem so that it can be used on the main app. I think it works quite well for us so far.

more info: http://stackoverflow.com/questions/6118905/rails-3-1-engine-...

When they say gems, I would have to assume they mean rails engines since they mentioned they are using it to package up assets as well.

I've been using the same pattern. It works ok, but still feels clunky. ∞ bundle update. dev mode reloading for certain classes can be a major pain as well.

We do use a Rails engine for one of our components, but this isn't something we've explored a lot (or at least, I haven't personally; maybe some of the other Airbnb nerds will step in with their own thoughts). Even so, I don't see this as a good long-term solution for our architecture, mostly because not everything we do is Rails -- we currently run JVM services and Node.js services, for instance, and we'd like to keep things flexible in the future so we can always pick the best tool for a particular use case. Our routing service should be able to handle all those needs.

I'd love to know more about the service discovery system and whether it's going to be open-sourced.

Was the implication that it's a layer on top of ZooKeeper or something completely different?

Hey there, thanks for your interest!

Our service discovery system is indeed a layer that sits between ZooKeeper and the application. But it's also pluggable -- that means that you don't have to use ZooKeeper as the system that tracks which service nodes are available; you can use something else, and write a plugin for our service discovery system that will talk to that instead. So it's not necessarily a layer on top of ZooKeeper, although we certainly use it that way.

I don't want to say too much about the technical details yet since we're still rolling it out in production and may change some pieces before it's released. Also, I'm not one of the primary authors and don't want to steal their thunder. ;) But yes, we do plan to open source it once the code is fairly stable (and once we have some time to jump through all the usual hoops of open sourcing code), and then we'll do a detailed post about how it all works. It's really cool and stunningly simple.

Reply on news.ycombinator.com