Hacker News Reader

Comment by zemanel | Hacker News Reader

Did you take a look at pip-tools ? It allows dependency locking

What do you guys mean by dependency locking? What I do with pip is add a requirements.txt with things like:

Django~=1.10.0

And it keeps the env autoupdated with the latest patch revision but not the latest minor revision. pip-tools doesn't seem to do much related to that, no?

The problem with that is that you no longer have reproducible builds. If you checkout a year old commit and built deployment artifacts with it you want the resulting artifact the be the same as when it was created. This is what you get with Bundler and the Gemfile/Gemfile.lock split

You put

gem 'rails', '~> 4.2.7'

in your Gemfile and when you run `bundler install` the exact version of rails you ended up resolving with the above is locked in your Gemfile.lock. Thus if you have a version in Gemfile.lock it is the source of truth and you have reproducible builds. If you want to update rails given the pattern in your gemfile you just issue `bundle update rails` and it will update to the latest conforming version and update your Gemfile.lock

The fact that pip doesn't support this in a good way is a major limitation imo. You can have either reproducible builds or "auto upgrading", but not both. pip-tools solves this, but requires you to use a different tool than pip and by extension you have to teach and explain it to anyone working on your project.

I version my docker images. Inside the Dockerfile I usually install the latest version (i.e. pip install without specifying version). By single version bump of my image version I can trigger update of everything.

If I check out an old commit I can look up docker image version in the source control, download exactly the docker image that was running at that time and start shell in this image. Inside the image I can run "pip freeze" to check exact version of each package that was running at that time. I can then update the old Dockerfile with those specific versions by adding the "==x.y.z" e.g. "pip install oauth2client==3.0.0" in the pip install to reproduce the old build.

Additional benefit is that such scheme works for system packages, or for any other language libraries I can think of.

Yeah that's what I mean by either reproducible builds(e.g locking each dependency to an exact version. Output of pip freeze) or what the GP is talking about which is specifying a version pattern.

> You can have either reproducible builds or "auto upgrading", but not both.

Ah, okay, this crystallized it for me, thank you. Other commenters were talking about one or the other as a limitation, which confused me because pip does do both (just not at the same time).

Yes, this is currently a limitation, and pip-tools seems to solve it. Hopefully pip will get a native solution soon.

I always though the recommended way to handle that split was by using setup.py as the Gemfile equivalent with very loose versioning rules, and for the Gemfile.lock case to not edit requirements.txt but to only generate it from pip freeze (or whatever it was - I haven't used Python for a while).

Yeah I've heard people say this as well, but from my understanding setup.py is more aimed at libraries than end user projects. I¨ve never seen anyone use setup.py for an end user project in any case.

Interestingly enough in Ruby it's the reverse, you use gemspec for gems which doesn't have the concept of locking. Instead you are supposed to specify semver conforming version patterns and then resolution of these happens when the gem is installed. Only end user projects use Gemfile.lock

Reply on news.ycombinator.com