Instead of rewriting your supervisord config every time you should be able to use a symlink that points to the newest virtualenv. I haven't tested gunicorn, but reloading uWSGI will resolve the new symlink to your new virtualenv. Rolling back is simply adjusting the symlink.

Also make sure you're on pip 7+ to take advantage of the automatic wheel cache to speed up building new virtualenvs.

As another commenter noted, I discussed a similar approach at DjangoCon US this year https://www.youtube.com/watch?v=SUczHTa7WmQ&t=1083