Hacker News Reader

maybe I misunderstood your comment, but it's pretty straightforward to get the exact version, if you do

  pip freeze > requirements.txt

it will have all the files' versions specified

That's what I meant by

generate a second requirements file after having done `pip install -r requirements.txt` to get the exact versions that were installed

So you'd need to have a `requirements.txt` with loose versions suitable for upgrading your apps deps, run `pip install -r requirements.txt` and then `pip freeze > requirements.locked.txt`. Then everyone should be using `pip install -r requirements.locked.txt` as well as during your build. But that's cumbersome and error prone and doesn't free you from having the wrong version of a dep in case you `pip install some-package` later on.

I'm not sure why you'd need two requirements.txt. You'd normally create a virtualenv, pip install what you need, then lock the versions with "pip freeze > requirements.txt". You don't need an initial requirements.txt to install new packages.

You want to be able to distinguish between loose dependency versions and strict locked versions for deterministic builds.

What fphilipe is talking about is something like

    $ cat requirements.txt
    requests>=2.12.1,<3.0.0
    $ pip install -r requirements.txt
    $ pip freeze > requirements.locked.txt
    $ cat requirements.locked.txt
    requests==2.12.1

This way you can run pip install -r requirements.txt when you want to update your dependencies and then lock the resolved dependencies in requirements.locked.txt so that you get deterministic builds when the code runs in production environments where reproducibility and reliability are important. It also gives you a clearer idea of what are top level dependencies and what are transitive dependencies because the transitive dependencies will only be listed in requirements.locked.txt. However this system has limitations and isn't standardized. If you want to have different groups, say development, production, testing. You end up with

   + requirements.development.txt
   + requirements.development.locked.txt
   + requirements.production.txt
   + requirements.production.locked.txt
   + requirements.test.txt
   + requirements.test.locked.txt

And even if you can tell which are your transitive dependencies by comparing .locked.txt to .txt it does not tell you why a given transitive dependency is in your locked dependencies e.g you don't know which of your top level dependencies is pulling it in.

One common reason is avoiding defining hard dependencies to versions of your transitive dependencies. In my current Django project I have 19 declared dependencies and 26 transitive dependencies. We have one file for the declared one and then another we generate with pip freeze. This way the transitive dependencies can evolve on their own without us having to keep track of them.

Pipfile looks like a definite improvement over the pip install, pip freeze workflow.

>This way the transitive dependencies can evolve on their own without us having to keep track of them.

This sounds exactly the opposite of what I'd want. I don't want some one to slip in a Guy Fiery into my dependence chain without me noticing.

You're misunderstanding. The whole point is that nothing slips in, but at the same time, you don't have to force a specific version of something in order to achieve that. The killed feature of Bundler for long term maintenance is the ability to upgrade a single requirement in a minimal fashion.

So you start with a Gemfile that is your minimum requirements with no versions specified, the first time you `bundle install` it generates a Gemfile.lock which is then sticky. Over time your requirements are completely frozen until you decide to update, which you can do piecemeal via `bundle update gem1 gem2 etc...`. If you have a reason to avoid a newer library, then put a soft restriction in the Gemfile, preferably with a comment as to why that restriction is there and you have a very powerful long-term system for managing versions over time.

Just freezing and forgetting is a recipe for disaster when you have to update months or years later, and the transitive dependency updates are overwhelming and conflicted. Similarly exact versions specified make it fiddly to upgrade and hard to tell if there were reasons behind specific versions.

You will want an easy to upgrade file for your development environments. And you will want a known good file for testing and production.

It's nearly impossible to further develop the result of 'pip freeze'.

There are messy edges like installing a git repo not in PyPi.

And if you don't want to carry all dependencies, you can:

    $ pip install pip-chill
    $ pip-chill > requirements.txt

Note: I built it because I got tired of reading through long auto-generated requirements files.

Maybe I am missing the point, but "loose" dependencies go in setup.py, and "exact versions" go in requirements.txt.

https://caremad.io/posts/2013/07/setup-vs-requirement/

The model you describe works well enough for Bundler with a Gemfile describing desired versions which can be loose or tight, and a Gemfile.lock specifying exact versions for all dependencies. It works much better in practice than one without the other, as in the case of package.json and non-deterministic npm.

Why are you saying that? You can freeze npm deps if you want to. Frankly npm does something very right which is allowing diffetent versions of the same dependency in nested tree of dependencies. I dont think there is another language which allows that?

Cargo allows for multiple versions of transitive dependencies as well. We attempt to flatten as much as possible though.

You can hack it in a fair number of languages, but yeah, NPM's approach is pretty uncommon. E.g. in Java, you can use "Jar Jar Links" to recompile a lib into a new namespace, which can allow multiple versions to coexist. NPM does make it transparent though, which is extremely convenient, and I can't name any other language that supports that.

But that's not always an option. Bower exists for a reason - all that duplication / bloat is unacceptable for browsers to download. It can also mean hell for static initialization / mutable state, because there's no longer a single owner of the global resource.