Comment by maybird | Hacker News Reader

Comment by maybird | Hacker News Reader

Parent Full thread

  One other valid problem I've heard raised about git-based
  deploys is that you can end up with cruft in your working
  copy that sticks around like .pyc files where the
  original .py file is deleted and there is the chance that
  this file could still be imported even though the
  original .py was deleted.

I'm shocked. If a py file is modified, the pyc is rebuilt. So we know it's poking at the original source file. Why not fail to import if the original source is missing?

Maybe this behavior is meant to support binary-only distribution of Python applications, but there really should be an option to override this behavior.

Edit: Did a bit more research and found this:

(1) http://docs.python.org/release/1.5.1p1/tut/node43.html

  It is possible to have a file called "spam.pyc" without
  a module "spam.py" in the same module. This can be used
  to distribute a library of Python code in a form that is
  moderately hard to reverse engineer.

Makes sense.

(2) http://docs.python.org/using/cmdline.html#cmdoption-B

  If given, Python won’t try to write .pyc or .pyo files
  on the import of source modules.

So you can mitigate that behavior by removing existing pyc files and using "-B".

When I ran into this problem I simply added a post-checkout hook that deletes *.pyc files within the git root.

Same here. Is there a downside to this approach that I'm missing?

I can confirm, that one alredy bit me.

Removing existing pyc files when deploying sounds sensible, but I'm not sure if -B is really a good idea in this case, as each new worker process will have to parse the code again instead of just using the bytecode.

I've experienced this before. Not that .pyc/.pyo files ever get into my repo, but sometimes Python will decide it doesn't need to recompile the files and I get old behavior from a piece of software I know I've just re-deployed.

My solution? Simple, I've got a a Fabric command that deletes all .pyc files in the repo after pulling.

def clean_pyc(): run('find . -name "*.pyc" -exec rm {} \;')

And that's that :)

Look at "git-clean." If I were using git-based deploys with Python code, I'd definitely run that after checkout.

    export PYTHONDONTWRITEBYTECODE=1

may be more convenient than passing -B.

As said in another reply, you don't want that on your production server, pyc files are a useful optimization. You should delete all pyc files when you change three code, though.

so add them to your gitignore.....

The problem being described is not .pyc files being pushed in the git repository. The problem is on the target server, the python runtime creates .pyc files for modules as they are imported/used, and if a deployment removes some .py files, you would be left with the .pyc file unless you take (simple) action to remove it during the deployment.

of course, the perils of replying after skim reading!

as pointed out, hooks are a perfect fit for this.

Reply on news.ycombinator.com