Ginger Tech Stack (Realtime + Django) | Hacker News Reader

Ginger Tech Stack (Realtime + Django)

lincolnloop.com

93 pointsipmb22 comments

What hit me about this article is that I employ almost an identical stack and have never really stepped back and taken a look at just how many pieces are required to make a non-trivial web app. When you're working with these tools day-in, day-out, you get almost institutionalized and forget the complexities that you've absorbed over the years.

A side note: it's a testament to the great work the open-source community has put in over the past decade (two decades?) or so that so many pieces can come together and work relatively seamlessly.

Very interesting. I was agonizing over the weekend about which stack to use for a new project. I know Rails really well, but Express/Node.js is so hard to ignore these days. This solution seems like the best of both worlds, but I'm still not sure how it works:

In the described scenario, Django loads to initial page with the Backbone.js/Socket.IO code and Django continues to handle all post/put/deletes while Socket.IO delivers all the "get" requests via web sockets?

Yeah, I think you've got it.

1. Initial page load dumps in as much data as possible to get things started.

2. All writes go through the Django API (we just look for a success/fail response).

3. All new/real-time data gets pushed to the client via socket.io. This includes data that you yourself created.

#1 is kind of a given, but #2 & #3 are more personal preference/app-specific I think.

We could have done #2 through socket.io, but as noted in the post, Python is our primary language and we wanted to push as much of the logic to it as we could.

At first we had content creators reading responses from the API with all other team members getting their updates from socket.io. This was problematic because creators would get content twice (once over the update channel for the team and once from the API). It resulted in some strange race conditions and a lot of "have I already received this message?" code. In the end, it was easiest for us to just always defer to the data from the socket (which was faster as well).

This makes a lot of sense and I look forward to trying this out in my own project. Thanks.

What is your system for pub/sub from the client's perpective? Do you have matching models on Backbone & Django?

I guess what I'm interested in is how to know what users to route what data to via socket.io.

Great stack! We use something very similar.

The client passes a unique key with a short TTL to the Node.js server that maps them to the proper user and teams for the duration of their session.

I'm too am comfortable with Rails and the Ruby ecosystem but jumped into Express recently with Socket.io and MongoDB and think you should too. My curiosity was shortly satisfied and I decided to stick with synchronous code instead dealing with callbacks. For a moment there, I was high with "I'll use Node with everything!".

But the experienced inspired me to play with Ruby's EventMachine, Cramp, Goliath, and implementations like socketio-ruby on Cramp. I also took some time out of my life to really dive in to Javascript and experience some new tools.

Is using a queue for calling Node.js the better/proper thing to do here? I thought this was the exact use case of Thrift.

Thanks for the post, I wish more companies would describe their stack at least in a high-level way, so we can know what is working "in production".

I saw the Ginger video demo, where is any real-time functionality needed?

There were real time discussion updates in the video

As the product evolved, the real-time needs (surprisingly) became less than we originally expected. I think this will change as we continue to evolve.

That being said, after the initial page load, almost everything happens without a page refresh or full page load. You see new messages (with favicon notification), votes, unread counts, etc. all in real-time. I think Ginger would feel a lot klunkier without the real-time bits.

Have you considered Mongrel2 or ZeroMQ for your stack?

Not much more than a cursory look. It sounds like they aim to solve some of the technical challenges we encountered though.

Looks great, congrats on the launch Pete!

Thanks Frank! I certainly can't take the credit though. Almost everyone on our team (http://lincolnloop.com/about/) has committed to the project repo.

Did you try tornadio2?

AFAIK it was released after we settled on Node.js.

We use Tornadio2 along with Pika (with the IOLoop sharing) for an internal project and it's been great.

This is the first time I've heard of tornadio2; it turns out there is a list of other socket.io server implementations at https://github.com/learnboost/socket.io/wiki/

This stack has no soul.

This made me chuckle. Unfortunately most of the people on HN will probably take it the wrong way.

I get that you want to make a joke.

I also think you need to understand it's not a joke. The fact that there exists "kick a ginger day" [0] is proof that it's not a joke. Abuse is not funny, ever.

[0] http://laist.com/2009/11/24/kick_a_ginger_day_spawned_at_lea...

Reply on news.ycombinator.com