Justin.tv porting their site from Rails to Django? | Hacker News Reader

Justin.tv porting their site from Rails to Django?

djangohire.com

125 pointsplinkplonk59 comments

Hey guys. This was my call, so I guess I should explain. I'm typing from my iPhone, but here goes:

Our site is over 5 years old, and if you've followed Justin.tv at all, you know it's been pivot city. All of those pivots have left their mark on the aging code base. It also turns out that many of our assumptions about how to build and scale a high traffic web app are no longer relevant. Servers are 10x faster, bandwidth is dirt cheap, and SSD's have entirely changed the database world.

There is nothing wrong with Rails, but everything else at Justin.tv is written in Python, so I wanted to use that. Switching languages has the benefit of forcing us to remove stuff we don't need anymore and implement the minimum set of features needed; cut and paste doesn't work across programming languages the last time I checked, and that's a good thing in this case.

Django has come a long way as a framework, to the point where it no longer "gets in the way". But the main reason I like it is that it's simple and it's python. All of our backed systems and video libraries are in python, so it will be nice to be able to share more code and leverage the brainpower of the non-ruby programmers at Justin.tv. There are many.

One thing I'd like to make clear- this isn't one of those massive, paralyzing rewrite projects. Justin.tv's website alone is not terribly complex. The scope of this is limited to porting only the templates still in use, porting over the minimum set of features, and removing as much cruft as possible. Our API, chat, video, data stores, and 99% of the infrastructure are untouched.

If you want to help build what will quickly be one of the top 5 django sites in the world, I'm hiring. Team is just two developers right now (everyone else works on TwitchTV) and we're looking for a lead developer.

Sounds mostly reasonable, but would you mind explaining why you ended up using RoR if you are mostly a Python shop?

5 years ago it was four dudes in a two bedroom apartment trying to run a reality tv show. Web frameworks were the least of our problems, especially since none of us knew any Ruby or Python to begin with.

Well said, Kyle.

I've tried three times to switch from Rails to Django, but each time I ended up coming back. Python isn't bad, but it's a more asymmetric development experience than Ruby (IMHO) and simply doesn't sit as well with me.

However, Django is just so radically different in its approach than Rails. It may have the admin console baked in, but other than that I felt that Rails was far more "batteries included". It's nice that it leverages the Python ecosystem in many cases, but I found it frustrating to have to make so many decisions on what to use when Rails typically provided most everything I needed (minus auth and file uploading...but two gems quickly solves that).

I'm sure if I had learned Django first instead of Rails it would be a different story. As it stands, if switch to anything in the future it's likely to be Play! (http://www.playframework.org).

ymmv

Having learned Django first, and attempted to build stuff with Rails (for intellectual curiosity) multiple times, it's interesting how I have basically the exact same opinion as you in reverse.

For me, the Django way of doing (almost) everything makes sense. Rails feels backwards. I also feel like I get more baked-in with Django, even when not factoring-in the admin. And its distinction of "Apps" vs Rails' "Gems" makes me feel like I have a lot more available from the community in terms of third-party plug-and-play code. Being able to have a minimum viable product without writing a single line of my own Django code because of apps in the community just doesn't exist for Rails as far as I could tell.

what do you mean by "asymmetric"?

It's hard to know all the details but it seems like most other video providers* scale by using http caching [max age + etags] with a CDN. Language performance isn't really the bottleneck remote calls are much slower :/

I have never hosted/written a django app, do people typically run a request in a thread in django? I thought python had a GIL just like Ruby (1.9+).

I wouldn't recommend using something like rails to stream video, however there are tons of great asynchronous ways to do this and it seems like the justin.tv guys are familiar with this. I would be curious to hear why the rewrite?

* youtube: http://highscalability.com/youtube-architecture * itunes: I have a friend on the itunes team and from what i can garnish from him their java servers are single threaded and their boxes hum around 10% cpu usage however they are bound on memory (similar to rails as they are not using java threads for each request). Akamai serves most of the content as I understood it.

> I have never hosted/written a django app, do people typically run a request in a thread in django?

Depends on the handler, mod_wsgi allows for m:n (mixing threads and processes) for instance.

> I thought python had a GIL just like Ruby (1.9+).

That's correct, but any C IO (such as DB queries) will generally release the GIL, which allows other threads to run.

(technically, CPython has a GIL, Python-the-language does not mandate it and neither Jython nor IronPython use a GIL. Pypy also has a gil currently, but Armin has started exploring STM to see if it would be a good replacement)

> That's correct, but any C IO (such as DB queries) will generally release the GIL, which allows other threads to run.

Ditto with Ruby too.

Btw: http://highscalability.com/blog/2010/3/16/justintvs-live-vid...

Cool, thanks!

What is justin using for streaming these days? Wowza? Something they built themselves (I know they used a custom built python solution initially, but they moved to something else along the way). Are they using ffmpeg to transcode the high bitrate game streams to the lower bitrate streams for viewers with slower connections? It's been a long time since that high scalability article came out.

One possible reason for this change is that Justin.tv has long used Twisted (http://twistedmatrix.com/trac/) for their network engine, and Twisted is written in Python. I'm not sure if they're still using Twisted, but if they are there might be some advantages to using Python for their web layer as well.

It's possible but that probably doesn't mean integration is easy. Writing async code in python forces you to use the-twisted-way or the-greenlet-way. That is the problem with language / standard lib lack of support for async calls - that is part of the appeal of node.js.

I doubt just reusing the twisted code inside the django app makes much sense for that reason (nor does the other way round).

I wonder how they'll avoid the "second-system effect". (http://en.wikipedia.org/wiki/Second-system_effect)

Second system effect is a possibility. Not a mandatory trap for any team doing a second iteration of their product.

All it takes is for a single person on the team to be aware of the risk and to warn his colleagues.

I would also argue that Justin.tv has enough experienced engineers to avoid the SSE.

SSE is a kind of trap that preys on ignorant and inexperienced.

I wonder what the motivation is. RoR is more mature and it otherwise is a fairly lateral port in terms of features and speed.

Both are dynamically typed and tremendously slow compared to the right .net or JVM stack. That would be my main criteria for stack selection in a project of this magnitude.

> RoR is more mature [...]

Really? Really?!?

https://rails.lighthouseapp.com/projects/8994/tickets/1203

Marking a bug as "wontfix" because the developers of the RoR platform are arrogant enough to say, and I quote, "[...] figure out what other software is putting that version in your environment and stop it from doing so" is now called mature?

The dialogue on this bug alone was enough for me to put RoR in my rear-view mirror and not look back.

Which would you rather use: a tool which demands it is the only tool on your system which can set environment variables (RoR), or a tool which plays well with others (Django)?

"figure out what other software is putting that version in your environment and stop it from doing so"

I don't think you understand the context here. The Rails dev is saying (correctly) that nobody should be polluting the environment with a variable as vaguely named as "VERSION" this includes Rails.

Rails uses the "VERSION" environment variable temporarily for migrations. They're not trying to "claim" that variable name for themselves only.

I believe "mature" is a term used to indicate that software is farther along, has more mindshare, and whatever benefits that come along with that.

As someone who's worked with both Rails and Django professionally for years, Rails is far more mature. Both are quality frameworks, but Django has far more warts, and the quality and number of good third-party libraries really isn't close. Django has forms and the admin going for it, and South is really excellent, but on the whole Rails feels better thought-out and more flexible than Django.

Django is very strong as a rapid-development tool (forms, generic views, admin), but I am more comfortable with Rails for a project with a reasonable timeline. Django class-based views are a far cry from Rails controllers.

Wow, completely agree.

They just need to rename the environment variable into something like RAILS_VERSION instead, as suggested by several commenters in that issue. Such a simple fix, I don't understand why they won't do that.

There are two problems with your statement

1) VERSION is a horrible name for a non-transient environment variable. It's fine if you're just using it on a per command basis (as rails does with 'rake db:migrate VERSION=123') but some app has stuck a ridiculously generic word into a global namespace. 2) (trivial) RAILS_VERSION would make no sense because it's the migration number you want to set your database too.

Any non-transient variable that you need to set should be specific enough to not conflict with other apps using the same namespace.

If anyone should go fixing things it's the fool who permanently stuck a variable called VERSION in their users global environment hash.

I agree that there is a problem and that it is affecting rails users. But, if for example some whacko was running around slapping all rails users with large fish would it be the rails dev teams responsibility to stop him? No.

No development team should have to tailor their app to compensate for the stupidity of one ignorant developer who couldn't think about the ecosystem that we all have to share. The only exception to this is when the idiot developer got a job building your OS...

Meh, using an environment variable named VERSION isn't too bright, and if it's for a single command then it should be a command line argument. RAILS_VERSION may not be appropriate but it's more intelligent than using an extremely generic and common name like VERSION.

Not sure if they will gain a lot of performance from this transition...according to shootout ( http://shootout.alioth.debian.org/u64q/which-programming-lan... ) python is only few % faster than ruby. Maybe they should consider porting their entire ecosystem to JVM as twitter did it.

They could run the site on PyPy as opposed to the CPython interpreter. Postgres and Django are well supported.

https://bitbucket.org/alex_gaynor/pypy-postgresql

http://speed.pypy.org/timeline/#/?exe=3,6,1,5&base=2+472...

I would feel hard pressed to sell PostGres with PyPy as well supported at the moment.

Django maybe, purely because it doesn't depend on too many C Libs.

The thing about the programming language shootout is that the benchmarks are optimized to within an inch of their life, and in particular don't generate much garbage. This may have changed, but last time I checked (MRI) Ruby's garbage collector was _stunningly_ slow.

performance of a web app is quite likely more dependent on factors other than the raw interpreter speed, e.g. straight rack apps could easily be five times faster than rails apps using the same interpreter.

They could gain in productivity and ease of support

You're implying that python is more productive than ruby? I've never heard that claim before.

Really? I've heard both that claim and it's c.ounterclaim several times and come to the conclusion that it's an interesting yet ultimately futile discussion.

Not at all. You can be more productive in another language for reasons that have nothing to do with the merits of either language. For instance, the Justin.tv guys already use a lot of Python in their stack outside the RoR app server.

I am not implying that one language is more productive than another, just that it might be more productive for that team with their codebase.

It seems like it would be easier to rewrite the site in a newer version of Rails rather than switching platforms entirely, so it sounds like someone's skillset is more Python based.

Oh! Does the Python world have something like Rack?

Yes, there is WSGI which Rack is roughly based on.

WSGI 1.0, PEP-333: http://www.python.org/dev/peps/pep-0333/

WSGI 1.0.1, PEP-3333: http://www.python.org/dev/peps/pep-3333/

Rack was inspired by pythons wsgi

"We're located in downtown San Francisco and have a sick office overlooking Market St."

hope sick office is a typo, must be slick office

Also US colloquial for very awesome, good, etc. Just like 'sweet'. http://www.urbandictionary.com/define.php?term=sick

now I understand why natural language processing is tough, nuances, context and downvotes

For an even more extreme example, see some of the comments on this: http://www.reddit.com/r/australia/comments/kq3l7/how_bad_a_w...

Or perhaps they're really honest, and it has Sick Building Syndrome. http://en.wikipedia.org/wiki/Sick_building_syndrome

It's been used here in the UK too since as far as I can remember

You mean their office is "the shit"!

I switch from Rails to Django a while ago. On a feature base the two frameworks are equals. I explained it in depth in this video http://vimeo.com/31083901 ( It's in Italian,mf.Du you speak it?! :-) ). Basicaly I turn on django beacuse I prefer Python to Ruby: again the two languages starts from the same assumptions ( Guido also told that ) it's just my personal taste. Also django do things THE WAY I like more. For ex: the configuration is simpler because it uses python files as config an not [name yours]ml. That said in DEVELOPMENT MODE ( production is a total different world ) Django is faster than Rails. The time to running test/ start dev server is not 100% of a developer productivity so I'm not saying that django is more productive than rails. I'm saying that I'm more productive with django than with rails. Use C++, Fortran ,Lisp ,Scala what ever you want: just deliver good software. PLZ

<quote> Use C++, Fortran ,Lisp ,Scala what ever you want: just deliver good software. </quote>

Absolutely

Minor grammar Nazi point (since you mention you are Italian, and might be interested) - "turn on" is either "sexually attracted to" or "suddenly attack", depending on the context. i.e. "She turned me on, the smell of cigarettes is not a turn-on for me, the dog suddenly turned on me and started biting". I've no idea why ...

You mean "turned to".

Then why does "...turned on a television" work?

Because it's a phrasal verb with multiple meanings: http://www.macmillandictionary.com/dictionary/american/turn-...

Right. The OP's comment made it seem like there are only two definitions of that phrase(neither of which applied to my example):

"turn on" is either "sexually attracted to" or "suddenly attack"

Yeah, I couldn't think of any other's. Phrasal verbs are tricky like that.

This comment thread was brought to you by the people who came here from /.

I've heard people say "he turned me on to the idea of..." without it having a sexual context.

Oh, the English language...

Not a grammar issue, it's a colloquialism. I used this phrase all the time.

"My old roommate really turned me on to the Wire. What a great show."

Thanks , yes I meant "turned to"... accidental humor :-)

Like putting roller skates on a turtle.

> I talked to someone from Justin about 2 years ago. He told me they wrote Twice to scale Rails because there was no easy way to make Rails fast.

Doing it wrong.

Reply on news.ycombinator.com