Must-have Django packages | Hacker News Reader

Must-have Django packages

devcharm.com

154 pointsrule3096 comments

Interesting, my fairly large django project uses none of those. Instead, the addons we consider to be critical are these these:

    django-debreach
    django-redis
    jinja2
    pytz

Redis for queues and caching, debreach to secure the CSRF tokens and vary your content length to avoid the BREACH vulnerability. jinja2 for doing templates outside of returning data, and pytz so you can use timezones in your data models.

I also recommend installing `flup` and running via fastcgi, but that's open to interpretation.

Does django-redis block? I'm not sure I'd make a connection from django to anything other than a low latency queue.

I think Redis fits your definition of low latency queue. It does in my book, and I run a few thousand concurrent connections to Redis from Django. I also endorse django-redis and use it for caching, pub sub, and various data types.

See https://github.com/samuelclay/NewsBlur/blob/master/settings.... for my redis configuration. I also use django-redis for caching (which is higher up in that file).

Yes, but then again so do the underlying mysql connections.

It does use native python sockets, so gevent.monkeypatch would probably work.

Blocking is one of the reasons I use flup and fastcgi - it doesn't matter if one thread of execution blocks; it won't affect any other requests. I've tested this under load with some really gnarly queries, and it works fantastically.

pytz probably doesn't belong in a discussion of Django libraries. It's needed by pretty much any Python program that works with localized datetimes.

  > jinja2 for doing templates outside of returning data

What do you mean?

I assume for things like templating emails. Or configuration files for deployment. That's at least what I've used it for.

It's handier and more straight-forward than using Django's templating engine.

I thought he might have meant that, but I never had too much difficulty doing this through the Django template API.

It's not bad, but the Jinja2 language is more expressive, and the API is a bit better. Not enough we would migrate away from Django templates to it, but enough to not migrate away from Jinja.

Fair enough, I tend to use something very restrictive like Django templates (I made/use an Erlang port of the language as well: https://github.com/oinksoft/dtl) or something very flexible like Mako if I have the choice and need it.

Amusingly, my must-have Django packages are mostly things that replace large chunks of Django (Jinja2 for templates, a DB connector/ORM for whatever database the project needs, which even for PostgreSQL is often plain SQLAlchemy).

I guess I'll just use flask.py or something for my next project. This is not meant as criticism, just my personal experience.

Yep, having everything built-in is a double-edge sword.

I myself love Django, though I find some parts of it a bit over-engineered (in the Java style). That is changing for the better in recent releases though. I'm thinking the render_to_response stuff and management commands, etc.

Have a look at Pyramid[1] if you plan a slightly larger project. I've been looking at it for years.

[1] http://www.pylonsproject.org/

Django guardian's performance is ridiculously bad. Better roll your own. We're using it and stuck with it at this point, but some of our page loads have thousands of queries because of all the permission lookups. Permissions probably need to be cached in memory.

Notable omission of South ( http://south.aeracode.org/ )

I think that's because South has been rolled into Django now. Is that correct?

Close, but not exactly, Andrew Godwin, the creator of South, has added support for migrations into the dev branch of Django, to be part of the Django 1.7 release.

https://docs.djangoproject.com/en/dev/topics/migrations/

Andrew has significantly rethought migrations and applied the lessons he's learned from years of maintaining and developing South.

He's kept a developer diary that has been pretty fascinating to read (YMMV):

http://www.aeracode.org/2013/11/27/new-apps-migrations/

and others at:

http://www.aeracode.org/category/django-diaries/

[update: route3 is faster than me, but still it's worth checking out the developer diaries]

For a greenfield app, would you recommend starting with 1.7, or do you think it will be simple enough to convert from South that it's not worth it? (I have no idea how stable 1.7 is)

1.7 is still in development, but see the expected release schedule[0].

  January 20th: 1.7 alpha  
  March 6th: 1.7 beta  
  May 1st: 1.7 release candidate  
  May 15th: Final release (assuming a second release candidate is not needed)

[0] http://python.6.x6.nabble.com/1-7-Release-Schedule-td5041311...

Andrew has said in the past that his intent is to provide a migration path from classic South to "South 2.0" which will backport migrations to earlier versions of Django (versions 1.4-1.6, I believe) and also build on top of the migration infrastructure in Django 1.7.

You should be fine sticking with South for now.

Cool, thanks very much to you and the other commenter.

Soon!

Migrations[0] are being included in Django 1.7 which is currently under development. For those unaware, Andrew Godwin, the author of South, ran a successful Kickstarter campaign[1] the fund this effort to incorporate migrations in Django core.

[0] https://docs.djangoproject.com/en/dev/topics/migrations/

[1] http://www.kickstarter.com/projects/andrewgodwin/schema-migr...

No, still use South everyday and will until it's rolled out in Django 1.7 - fingers crossed!

"Must-have" anything always feels like clickbait to me.

Eh, Internet is based on clickbaits and lolcats. $ad but true.

Just reminds me of scummy in-your-face ads everywhere, like at a mall or in a women's magazine.

Cripsy forms, because simple things like adding the necessary markup for bootstrap to forms is super-boring otherwise ; in general, it helps with custom form layout, which is usually a template maintenance hell

https://github.com/maraujop/django-crispy-forms

I've found django-allauth a bit friendlier than django-social-auth, and it covers registration of local accounts in one package.

Yep, an interesting comparison is found here: https://speakerdeck.com/tedtieken/signing-up-and-signing-in-...

https://github.com/omab/psa-allauth looks like it might have the best of both worlds.

Oddly enough, django-social-auth has been deprecated for some time now in favor of python-social-auth[1] (same author). It is more user friendly than django-social-auth, and also offers up a lot more functionality that django-social-auth wasn't giving me.

[1] - https://github.com/omab/python-social-auth

Everyone recommends the django-toolbar for spotting slowdowns in your SQL... ok, installed it. I almost never see any tips on how to actually do that though. What now?

(I suppose the SQL part of my app is not the problem though, rather the number crunching that happens on the data returned. I do lots of caching on the results instead to make the site seem responsive.)

I would say there are two primary ways DDT helps you spot database performance problems.

The first is that "oh my god" moment when you see via DDT that your view is doing 100+ queries to the database. This leads to research into Django ORM features like "select_related" and other ways to cut down the number of queries.

The second is clicking DDT's handy EXPLAIN button on a single query that is taking a long time. Sometimes this can help improve performance by adding an index to a field.

If you're doing a lot of # crunching, you might try using Cython for those bits. You can get a 10x speedup without changing semantics and just annotating types.

Thanks for the tip. Most of it is done by python-dateutil though so I think it in C already. I'm basically checking ~500 recurring events to see if they occur today with django-schedule.

Oooow. Interesting one. So the events can be on different schedules / frequencies? It sounds like something you should be able to do with a single sql query. Could you provide a bit more info?

Hmm, currently it uses python-dateutil to find recurring dates. I chose django-schedule at the time as it seemed the best choice, which is built upon it.

I'm not certain postgres for example has that kind of functionality around dates. Googling, I found this though:

    http://justatheory.com/computers/databases/postgresql/recurring_events.html

Looks promising. Rewriting the core of the app is a pretty big project though, it's just a side-project so I'm not sure I'll get that amount of time.

Just thinking that maybe you had a single core lookup that you needed to optimise. A slow running "find all events that repeat on today." If it's something that's more deeply scattered through the system then it might not be worth attacking.

If you just needed the events for today you should be able to do something simpler. Depends on the complexity of the frequency controls. But imagine weekly recurring - you should be able to do something like "(today - start day) mod 7 == 0" to find all weekly events that hit today. You could then construct a similar rule for each different frequency:

    -- not real code!
    select * from event where
    (frequency = 'daily')
    or (frequency = 'weekly' and (today - starts) % 7 == 0)
    or (frequency = 'monthly' and day(today) == day(starts))
    or (frequency = ...)

Just a thought.

I've been away from the Django ecosystem for the last few years.. how is Pinax doing these days?

Seems dead, a shame because I wouldn't mind a more complete starting point when creating a project.

What are people using these days for compiling their static assets (js minification + concatenation, etc)? We have been using django-pipeline but it has been a nightmare.

I use Jammit for static asset compression.

(from https://medium.com/cs-math/f29f6080c131) Jammit came out of DocumentCloud, and even though it was built for Ruby on Rails you can do something like this and use it for Django pretty easily. One of the things I like about it is you can setup different named configurations for javascript and css for different sections of your site (dashboard, non-authenticated, standard). It supports lots of other stuff too.

This is how I use Jammit on NewsBlur: https://github.com/samuelclay/NewsBlur/blob/master/utils/jam.... I then compress the assets, upload to S3, and then deploy to my servers. See https://github.com/samuelclay/NewsBlur/blob/master/fabfile.p.... Each app server then downloads from S3. So I can package 2MB of static assets and deploy to to a hundred servers in 10 seconds.

We're currently using Django Compressor [0], and it works pretty well. It was slightly annoying to get it to work well with staticfiles [1], as we host all of our content on S3 and serve through cloudfront in production. But, it's nice because now we can do this in our template:

    <script src="{% static js/mylib.js %}"></script>

and with compress like this:

    {% compress js %}
      <script src="{% static js/mylib.js %}"></script>
    {% endcompress %}

so that referencing static files is consistent.

[0]: https://github.com/jezdez/django_compressor

[1]: https://docs.djangoproject.com/en/dev/ref/contrib/staticfile...

I used to use this, and it is convenient for small sites. But it quickly gets hard to manage. It also makes it extremely difficult to share compiled assets between pages with different templates.

Why has it been a nightmare? I find it pretty neat. Was using django-compressor earlier but it didn't have support for Python 3.

I'm curious what problems he had with django-pipeline as well. I've been using it for a couple months now and it has been fantastic.

Caveat: Small site, low enough traffic that a few kilobytes per request won't break us.

I just enable gzipping and don't worry about minification of my own assets. Sure, it means larger initial downloads (since they're cached), but it's typically in the single digit kilobytes, and simplifies debugging like nothing else.

We switched to the google pagespeed module for nginx and stop worrying about it. It doesn't do minification very well but it does concatenation and takes care of a bunch of other stuff as well, like image resizing, CSS concatenation etc.

We used to use the HTML5 Boilerplate script but it was horrendously slow on EC2

http://brunch.io/ I've use it in Django, Flask, Rails and static sites with Chaplin, Backbone and now Angular. Works a charm.

We use django-assets. http://elsdoerfer.name/docs/django-assets/

+1 for django-assets. I've found this to be the best solution so far. I've used it combination with django-storages to compile and push assets to S3.

I'm pretty happy with django-compressor.

Me too.

This is a brilliant question, any package about that?

make (actually a simpler alternative).

nginx + pagespeed should be better way of doing it

What about South?

I agree South should probably make the list although database migrations have finally made their way into the Django core so perhaps that's why the author didn't mention South.

Have a go at editing the `Utils` section.

A few I work with on most projects are...

Backend: one of the most popular API Frameworks is Tastypie. Authentication and authorization: Django Userena. Debugging: Django Extensions. Django CMS Django Haystack

These are a must for your tool kit.

I would recommend DjangoRestFramework any day over Tastypie after having used Tastypie for two years.

DjangoRestFramework code base is better, it's better thought out, it should be part of Django core.

I've used both in projects I honestly don't like the auto documentation Django Rest Framework assumes you want. I enjoyed using both but I lead towards TastyPie, yum!

I currently use Tasypie.

Would I gain anything switching to Django REST framework?

We switched form Tastypie over to Django REST Framework, and are overall pretty happy. Cleaner, clearer code that's easier to work with.

Plus, it generates a browsable api that you can explore via the web, without any extra work. Which makes life a lot easier, believe me.

Bonus points that Daniel Lindsay (tastypie author) got a job with Amazon, works on boto, and does very little with tastypie anymore. D-R-F is actively maintained and is a bit nicer to work with. Also, the autogenerated apiview "self documenting api" is super duper nice.

Also, the DRF author, Tom, actively attends meetup and does talks here in Brighton.

The APIs are more sane. Added bonus of a cool HTML api explorer view.

How do you square that with the article? It specifically says that that Django REST is "insane".

Don't be a pedant. The author uses the word in the sense of the slang expression "insanely great."

Don't be a joke pedant? The author (me) uses the quotation in an ironic manner.

The word insane does not appear on the page

It previously appeared, with regard to DRF, where the word 'excellent' does now.

Wow, man. That's a lot of boilerplate.

"Django REST framework is an insane framework"

I'm fed up with this kind of oppressive language. This is heteronormative and is discriminatory to the mentally ill. It's stuff like this that leads to the imbalance and under-representation in the industry. Let's boycott this author.

(I would love not to have to write "</satire>" but having seen some of the comments here, I think I need to. I wonder how many white-knights-for-women use language like this without thinking. We need some flexibility in the way people are allowed to express themselves.)

Bah. Your comment is probably one of the less informative ones here, so I downvoted you. Look, there are turns of phrase that shouldn't be used by responsible members of certain communities in public. Saying something like "watch out, woman driver" is pretty offensive universally. We can all benefit from a more inclusive and positive language.

However, the original quote of "Django REST framework is an insane framework" sound innocuous enough to me. The problem I have with it is not that it's discriminatory towards the mentally ill. It's that it makes the author sound like a 13 year old kid. They could improve the sentence by using words like fantastic, great, indispensable, useful, etc. Saying that a "framework" is an "insane framework" is really kind of stupid IMHO. The author could have just excluded that meaningless sentence and the article would have been improved as a result.

Edit: also, searching through the comments on this page, you are the only one bringing up the "insane" word and making (bad) puns around it. Your original comment sounds like a reply to something someone did, except nobody here did anything like what you are accusing them of. Perhaps you might want to take to Twitter or Reddit for pointless venting at nobody in particular. </satire>

How is this heteronormative? I didn't see any reference to gender in the entire article.

Also, you're really stretching the "discrimination to the mentally ill" thing to the point of silliness.

I was going to say "the equivalent of heteronormative but for mental illness" but I thought that anyone that understood what I was saying might have just let me have that one slip and interpreted the meaning rather than focus on its strict representation.

It wasn't a joke as such, at least its intention wasn't primarily humour. Just highlighting the fact that nearly everything everyone says is offensive or exclusionary to someone and that if we adopt the universal application of the principles of extreme gender-neutral language that we've seen advocated on HN and elsewhere, equally for every issue, we end up with lobotomised language†.

I think the use of 'insane' is fine. And I think the use of gendered pronouns is fine. But I am neither insane nor of a gender that is under-represented in the pronoun

† oh never mind

This article has no relevance to gender-neutral language and no one other than yourself is discussing this. It would be better to address this issue in a relevant thread instead of hijacking a random one.

Specifically the casual use of the word 'insane'. I don't think it's unreasonable to discuss the contents of the page and make references to things not contained on the page.

I think you're thinking of the word "retarded". You might as well be arguing against using the word "crazy". Insane and crazy have identical meaning in most contexts.

Ableist is the word he was looking for.

should have used a bigger </satire>...

Or perhaps waterlion should have just not made the joke. It does not relate to the article in any fashion I can find relevant nor does it add anything to the existing discussion.

I think it adds to the broader discussion that goes on on HN, specifically over the last month or so.

Could you remove that joke about the dog from your profile page please? I went there looking for serious information.

If you could tell me what you're looking for instead of what you're not looking for, I'd be happy to provide it.

I'm looking for interesting discussion with intelligent people on a range of subjects guided by user-submitted stories. I'm looking for explorations of ideas, learning new things, synthesis as well as derivative ideas. Open-mindedness and acceptance. Otherwise there's no point.

I'm not looking for pedantry and closed-mindedness (I'm not accusing you of those things).

I'm probably in the wrong place for that on HN, but the articles are interesting and often so is the discussion.

That's an odd thing to look for on my profile page.

I'm going to give you the benefit of the doubt that I would want others to extend to me and not accuse you of pedantry. But I don't think there's much steam left in this conversation.

I think </trolling>

I was not trolling by the definition I know. I just thought that it might lead to an interesting conversation.

It's just too hard to communicate on the internet, especially with people who don't share your conversational goals. I think I just found a new year's resolution.

Let's Google "trolling". Definition of the informal use of the word:

"submit a deliberately provocative posting to an online message board with the aim of inciting an angry response."

Message board? Check! Deliberately provocative? Check! Angry responses? Check! Must be trolling.

Judging by your history on this site, you've done nothing but trolling.

It's a shame you think that I'm trolling. You're welcome to go back over my comment history and look at the conversations I've engaged in. Sure my comments provoke a range of responses. It would be a boring world if everyone always agreed with everything. Maybe the writing style isn't to everyone's taste. But they're never intended to provoke anger.

My original point was that the absolute application of principles designed to bring about equality could be harmful and that we need to apply contextual understanding to language. If the expression of that idea makes you angry, that's fine, but it's not trolling.

Anyway, I'm bowing out of this thread as it doesn't look like pursuing this idea is constructive.

At a minimum you chose an odd target for your satire. There isn't really any adjective that can be used in that location that will add much information to the text (it is just expressing enthusiasm).

An experiment: Next time you reach for some euphemism for mental illness, consider a couple of alternative phrasings (that do not use the euphemism). Decide if the alternatives are clearer or more precise. I don't run around taking issue with word choices, but if I run that experiment, I usually choose one of the alternative phrasings.

sounds oppressive but surely not meant this way - the author's mother tongue is probably german and in german "wahnsinnig gut" is a common idiom. It is just bad translation.

Terrible joke/10

It wasn't primarily a joke/10.

Reply on news.ycombinator.com