Django Production Deployment and Development Using Git | Hacker News Reader

Django Production Deployment and Development Using Git

jeffknupp.com

73 pointsjknupp25 comments

The global settings file is the single biggest hurdle, not only in django deployment, but also development. And it's been like this since I can first remember with no visible progress.

It sucks to keep track of dev/prod local files, it makes it hard to setup a simple script inside of a django environment, it makes multi tenant apps unnecessarily complex, app developers usually want to shove every single configuration option inside of it...

If anyone with more than a few hours of django development is doing split settings or something more complex, and even the official docs and best practices recommend doing so, why are we still stuck with it?

There are much saner ways to deal with configuration: Flask's[1] is one nice way, as is Padrino's[2].

As a Django dev, this is one of those things that actually makes me jealous of Rails, where the core devs are not afraid of breaking backwards compatibility for sanity's sake.

In my opinion it should be priority #1 for an overhaul.

[1] http://flask.pocoo.org/docs/config/ [2] http://www.padrinorb.com/guides/mounting-applications

I... don't understand your problem. Is it a PITA to split settings files every single time you start a new project? Yes, sure. Does it take more than five minutes to do so? No.

Really, Flask's config options are better, but they aren't that different from Django. Use the default settings, tell Django which settings to use from the commandline or specify an environment variable pointing at the settings to use. Where you put those settings is completely up to you - nobody's saying they absolutely positively must be part of your repo.

When you're working with multiple environments you'll have to specify which configuration to use one way or another so I'd prefer it to use the least amount of abstraction.

If you, me and anyone who's spent more than a few hours with Django split settings.py into multiple files (and heavily modify it), why is it still shipped as one monolithic file? Isn't the framework supposed to make our lives easier?

Splitting settings into multiple files is the least of my problems (nowadays it is not a problem at all).

The biggest problem I have with it is that it is global, which means I cannot safely modify it at runtime.

Why would you want to modify settings at runtime? Not that I'm saying there's no good reason to, I just don't know.

> As a Django dev, this is one of those things that actually makes me jealous of Rails, where the core devs are not afraid of breaking backwards compatibility for sanity's sake.

I work on Django since about 0.9-1.0 and I remember some settings (recently cache, static media) breaking compatibility, so it's not the case. IIRC a new settings module may be introduced in 1.5

and OP's workflow can be automated with fabric.

Well, I use Django since 0.95. Still, how long has it been since 0.9-1.0? 8 years?

Yes, I already use fabric for deployments.

I think there are perfectly fine ways of dealing with it in Django, but it sucks that the documentation does not recognize how vital managing your config is and doesn't expand on the matter.

Using a local(_)settings.py feels more esoteric than basic, because you have to find your own blog posts explaining how to do it. Having to come up with your own SECRET_KEY generator for your FOSS Django project feels hacky in a bad way.

I don't know if Django has an implementation problem, but I'd say it has a documentation problem.

I've been doing roughly this for a little while now, and it works pretty well. Particularly the bit about splitting the settings file. There are a few ways to do that, here[1].

[1] https://code.djangoproject.com/wiki/SplitSettings

At Mozilla, we use a homegrown template called Playdoh[1][2] that includes split settings file, some sample deployment scripts, and lots of useful libraries that handle stuff like secure cookies, localization, strong password hashing, etc.

I'm not super-familiar with our deployment process, but I do know that we have internal git repos that hold our production and dev environments (like local settings) and use puppet for setting up servers.

[1]: Github: https://github.com/mozilla/playdoh

[2]: Docs: http://playdoh.readthedocs.org/en/latest/index.html

Puppet is pretty cool. I recently used it to do provisioning in a Vagrant[1] VM to duplicate my development environment for other members of my team. I'll probably use it for production stuff soon, too.

[1] http://vagrantup.com/

I am learning django/web development, and will likely be launching my app in the wild soon. A link that details out how to manage deployment and source code management using git would be tremendously helpful.

Is there anything from the post you'd like more detail on? Email me at <myusername>@gmail.com and I'd be happy to go over everything I've done so far regarding deployment and source code management via git.

Thank you! I will be in touch.

This article by the Heroku guys also does a good job of explaining the core concepts of a modular approach to writing your app: http://www.12factor.net.

We manage the deployments of our Django application, in multiple environments (production/staging/experimental/development) using Buildout. It probably is quite difficult to learn at first, but very rewarding once you get the hang of it. Deployment is all about being confident. And the confidence that Buildout has given us, is uncomparable to anything else.

A cool way to produce various settings/configurations, is using the Buildout recipe collective.recipe.genshi. This enables us to create settings based on templates, using buildout configuration parameters. Very flexible solution!

As a Django developer/sysadmin myself, I'd be very interested to hear more details about how you deploy.

Well, like it says in the post, my root directory served by Apache is just a clone of my git master branch. When I've finished making changes and committed them to master, deployment is a simple "git pull" and Apache reload via postinstall hook. Sometimes migrations need to be run, but they're all staged via another postinstall hook.

Hmm, have you actually got everything working that way? I've found there's a few too many tasks to do that way, and wrote a Fabric script to deploy. It pulls git, checks for dependencies with a requirements file, compiles translations, runs collectstatic to put static files in the right places, and reloads Apache (and could run tests). Works quite well.

How does Fabric compare to setting up a git hook on post-merge to run collectstatic, etc.?

Not too big a fan of git hooks, but mostly because I haven't played around with them too much. My script is here: https://gist.github.com/1755101 A bunch of simple functions, some wrappers for directories and the environment, and an order of operations

Have you looked into things like Fabric for automating installations like this? It sounds like you're only deploying to one (or a few) machines, or is it more?

This part of the development cycle has always interested me, especially to see how other people do things.

I use Fabric for my deploys. The process is something like:

* Check out to clean directory

* Run tests

* Zip source

* Upload

* Unzip

* Backup existing (in case of needing rollback)

* Deploy static files

* Run South db migrations

* Restart

It's very easy to get started with Fabric and once you're using it, everything can easily be automated.

Cool! I use Puppet at work currently to do deployment stuff, but I've always been curious about other technologies like Fabric, et al. I'll probably look into them for personal projects or just for fun, eventually. It's on my [long and growing] list of things-to-do.

I use fabric as well with your same approach but I sometimes find easier to have git installed on the server and just pull from the main repository. I also use fabric for nginx and Apache config files so that I keep track of the changes and replace the files on the server if needed. And finally, I use fabric again to install needed virtualenv requirements.

I've looked at Fabric but decided that, for now, it's a bit heavyweight for my needs since I'm only deploying to one machine. If my deployments get more complicated, I'll definitely take another look.

Reply on news.ycombinator.com