Ask HN: How to automate Python apps deployment?

1) Look into configuration management. Ansible and Salt Stack are both python-based so everyone should be able to contribute if needed.

2)It's popular to run your automation through CI tools like Jenkins because it has auditing and logging built in and it's easy to set up.

3) Don't run bower/npm install/update on production because if npm or github go down then your production site will break because a partial code change has taken place. You should build (and test) on a separate server and then copy the code changes over to production.

Really you're looking for a Capistrano for python apps. If you're going to build a deployment tool on your own it's worth looking at their approach. For python, fabric is a fairly standard way to automate code deployments. The good news is that you're on the right path. Having automated deployment procedures will give devs a lot more time back AND remove a whole class of errors

I've moved my current company's deployment from manual line by line ftp deployment to an automated build and deploy using Jenkins to trigger ansible playbooks. Given the choice, I'd probably just use capistrano or fabric instead of building from scratch. I'd estimate that we've saved several months of work per a year in both deployment time and in debugging production issues.

That's one comprehensive list of items, thanks for this.

I will take a look at Capistrano, Fabric, Ansible and SaltStack, in addition to the configuration management topic.

I think your advise of copying source packages and dependencies makes total sense, I have seen it happen at other places.

My question remains, how would I automate a developer's environment setup (machine)? Assume I have the OS installed and connected to the internet already. Can I use the tools you referred to for deploying these apps to developers machines? What would you say regarding Python's virtualenvs creation? Can that be incorporated into the configuration management tools?

Adding to "Don't run bower/npm install/update on production" - a lot of package managers have a local mirror available and you can point your build server to it. This gives you some redundancy if the public mirror goes down, and also faster downloads within the network. Bandersnatch mirrors all of pypi, there are some others that will just mirror the parts of pypi you need and fetch new packages as you try to pip install things that don't exist in its local cache.

You can use Vagrant and Ansible, I think it's a killer combo, ansible provisions your vagrant/virtualbox machine for the dev.

I could integrate my entire workflow with Jenkins and custom parameters and custom development of console applications ( creating automated versions of packages ( npm / nuget / .. ), mostly with the use of the "build number")

It handles 3 situations:

- local ( not included in Jenkins, but is automated within the development tools -- eg. Visual Studio. It uses local databases and local paths)

- development

- beta

- live / production

So my entire workflow is automated now, the merging of branches and etc. is done by the developer. Testing is automated ( unit tests) ... Deploying is just a matter of minutes instead of 1,5 hours. And best of all, all related packages are updated automaticly with the correct tag ( development, beta, ..)

Another way would be using TeamCity ( instead of Jenkins)

Thanks for writing this.

We use Python virtual environments to make every app's dependencies in one place and not installed globally. Also, the frontend dependencies and code compilation is handled with npm, Gulp and Bower.

The thing is the developer has to issue all these commands in order to "setup" the app locally and be able to run and troubleshoot it before doing actual work, and this is universal for all apps. Same thing goes for setting up apps on staging and production other than the pull-requests and branches-merging tasks which are all manual.

And if a new developer comes in, they will have to install and configure their workstation before doing any actual manual setup which is also an overhead. This scenario would also happen if a developer comes back from a long holiday.

Any ideas on this?

For deployment: use your OS's packaging system, for both dependencies and actual applications. `git pull' rarely does good job, but `apt-get install' often is just enough.

For development: use your OS's packaging system for installing dependencies and a makefile for compiling things.

Really, I don't understand why programmers tend to avoid using sysadmins' tools in their work, especially now with this DevOps thing, when programmers are supposed to see some administration themselves.

I agree with you, I kicked off this workflow implementation phase with a simple bash script to see what parts need special knowledge and are ugly and not standard and then pushed back to the team to fix them. Using a bash script to automate the development environment setup sounds good but I thought I might have missed something, hence the question.

Do you think this is a good way to automate the boring workflow:

* git clone project

* make virtualenv

* edit config file

* npm install

* bower install

* exectue gulp build task

* enable apache internal url for project

* restart apache

?

> Using a bash script to automate the development environment setup sounds good [...]

No, it doesn't sound good even remotely. This way you have method for setting up things that is very brittle, probably can't be run with confidence on a machine that was already set up, and this method is not suitable for production or test deployment, so you need to maintain two setup methods.

You should look at how developers of various distributions (Debian, CentOS/Red Hat, Fedora, Ubuntu) package their software. All these foo-dev packages containing static libraries and headers, and foo packages with actual libraries and tools, either having their respective dependencies (e.g. foo-dev depending on gcc, so you'll get compiler by just running `apt-get install foo-dev').

Apache is not a good choice for development tool. It's not pretty to set up a second instance that runs without root privileges. I use uWSGI for those, I just use different communication protocols for production (FastCGI or uWSGI's own thing) and development (HTTP, to avoid setting up fully blown web server).

Running `npm install' or `bower install' or `pip install' to just set up your environment is a method for a big stall in case of network outage (see this thread: https://news.ycombinator.com/item?id=10984775), unless you prepare local caches for all these repositories. You should have all the things in your local repository, either source control repository (your stuff) or packages repository (external dependencies).

Can you elaborate more on the last paragraph? Also what do you exactly refer to by package repository and how would installing pip, bower or npm modules become different?

> Can you elaborate more on the last paragraph?

There's not much to say. You depend on external services (and network connection) to write, build, and deploy locally developed things. All this fails badly when one of the services experiences problems.

> Also what do you exactly refer to by package repository and how would installing pip, bower or npm modules become different?

Dude, really? `apt-get' didn't give a hint?

And all those modules are different from one, single package repository in that you won't install Python modules using npm, or JavaScript with pip, but you can do all those with `apt-get'. And you'll get repeatable process, since you control your package repository.

I am either lacking some experience with apt-get or you're not clarifying your workflows enough.

How would you clone a git repository, compile the SASS into CSS, uglify the javascript, and run a verticalization gulp task, after of course installing the frontend libraries through npm and bower all in apt-get? Oh no hold on, you won't use neither npm, nor bower to download the frontend deps... oh no, not even pip. Mind ``explaining like I'm 5``? :)

Apparently you lack training in using your server system if you thought that binary packages system is to be used for running compilation. No, it is to be used to distribute, download, and install pre-compiled (e.g. minified) files. Also, to distribute build tools.

For more explaination how package systems work, ask sysadmins in your company, as HN comments is not an appropriate medium for tutorial about them. If you don't have a sysadmin, well, you need to employ one anyway.

And this is exactly what I meant when I said that programmers tend to avoid using sysadmins' tools.

Apparently you made me think of something else, I managed to do something similar to what you explained in a different workplace, but it seems that it needed to take forever to make you clarify your point. Anyway, thanks for coming back and writing, I really appreciate it.

Longer blog post here: http://dna.hioscar.com/2015/10/14/scaling-deployments-at-osc...

We work out of a mono-repo and use pants (https://pantsbuild.github.io/) for python/jvm/js packaging to deploy on mesos (https://mesos.apache.org/). Pants gives us (effectively) the same development workflow as in production and let's us treat any application (we have ~500 at this point) as just a command line executable (for things with weird lib dependencies, we do statically compiled wheels for each architecture - CentOS and OS X in our case - on our own pypi mirror). We've got a few kinks to still workout for local development of web projects (mostly related to JS/CSS) but for things that don't touch the frontend it's pretty beautiful.

I am currently finishing up writing a step-by-step tutorial on this very topic. Please check it out and, if you have a question about any piece of it, write a comment and I'd be happy to help.

https://amfarrell.com/saltstack-from-scratch/

Also let me know if any of the integration tests for any of the tests fail on your setup.

Thanks for sharing this.

I will definitely check it out soon, but since we're doing releases and are short on time, mind explaining the general headlines of this tutorial? Say if I do everything manual from development machines setups and installation of deps and packages, to staging and production releases, what will I be able to automate after applying your solution? And what steps won't be covered? Can you give me a detailed list of these items?

You mentioned above that you were planning to check out SaltStack, this tutorial will give you working understanding of it.

Development machines setups: The tutorial walks you through running your app on virtualbox machines the run locally, so that is an option. However, it doesn't cover how to set up a development environment outside of a virtual machine.

Installation of deps and packages: This it covers thoroughly.

Staging and production releases: It does not walk you through this explicitly, but once you go through the tutorial, using the tools it teaches to do this is fairly straightforward.