Hacker News Reader

Comment by nubela | Hacker News Reader

I don't understand the need for this. The main benefit of the Pipfile seem minute, and can already be achieved with requirements.txt.

But I'm sure I'm missing something. Please feel free to convince me I'm wrong :)

View on HN

I'd like something like `npm install --save Django` -- a command that finds the most recent stable version of Django, and then adds _that_ package to requirements.txt, and leaves Django's dependencies out of reqs.txt.

I know that it's just a few simple operations for me to do it myself, but npm has spoiled me.

If I'm reading you correctly, the missing feature in pip is to list dependencies for one package so they can be put into requirements.txt.

This is a missing feature, there are various hacks but nothing concrete.

If you're working on a large project you have to keep track of all your dependencies and preferably only add the main dependency, not its dependencies, into requirements.

I tend to avoid doing pip freeze > requirements.txt because it lists packages I know shouldn't be in requirements.

I think you misunderstood.

Like civilian, my main ask for an improved pip would be a way to save a single explicit package into my requirements.txt by adding a --save flag when installing it.

I don't want to include sub-dependencies in my requirements.txt.

I believe I follow.

The way to maintain requirements.txt with single dependencies (base dependencies maybe, I'm self-taught on the lingo), is to manually add them one by one.

Pip freeze is useless because it adds a lot of things you don't need into requirements, making it cluttered and hard to follow.

Yes, I (and civilian) know that. The point is that it's comparatively cumbersome. You have to manually add the specific version into requirements.txt instead of being able to automatically save it as you install.

package.json makes so much sense to me, npm has spoiled me also

The main reason this is better is explained in the third paragraph in the README: Deterministic builds. You can specify target versions in your Pipfile, while your Pipfile.lock will contain the actual exact versions pip installed (even the exact git commit) so if your app is built on another machine you know the libraries are exactly the same, preventing slight version differences from causing you problems without requiring you to be ultra-specific in your dependencies.

Another benefit is named groups, which allow you to more succinctly specify dependencies for various environments (dev/test/production/etc.) Along with this you get the benefits of the lock file so you can be assured the subset of libraries you use in production will be the exact tested libraries you use in your larger dev/test environment.

deterministic builds are good, but i think the issue people are seeing with this is that it's like setup.py, in that it's not easily parsable without the full python interpreter. We could already have a requirements.lock.

in fact, there's already a library for this called pip-tools (https://github.com/nvie/pip-tools) that generates a requirements.txt (the lock file) from a requirements.in (your direct dependencies)

Worth pointing out that the builds will be deterministic only in regards to Python libraries. For anything else (e.g. libxml) different machines might still have completely different versions, and behaviors, so determinism goes out the window.

To fix that, you'd need something like Nix.

How is the Pipfile.lock distributed? Is it intended to be checked into source control alongside the Pipfile? If so, how would that help someone pip install my Python project (say from pypi) using that Pipfile.lock and get the benefits of a deterministic build?

Yes, source control.

Installing the project would cause it to be built from the lock file (assuming the Pipefile hasn't changed). This will mean your users won't get "some version" of a library you depend on between verion 1.0 and 2.0 or whatever range you specified. They'll get the exact package you last successfully used and checked in yourself, right down to the git commit if applicable.

Once you modify the Pipfile then pip will resolve your dependencies and try to make the specified changes by adding/removing/upgrading packages. If Pypa continues following bundler conventions this will be done by making the fewest changes possible from your existing versions in your lock file. You'll also have an upgrade mode where pip will rebuild you project from the Pipfile looking for the most recent versions of all libraries or a specified library within your specified version ranges. When done and your app/tests are working, check in the new version and you can ensure all your users/environments will be able to upgrade cleanly.

Depends on the use. For a library, it doesn't need to be distributed at all, since the application that's using the library will eventually dictate the actual versions of dependencies (so it can play nicely with other libraries). For an application, yes: you commit it along-side the pipfile, and re-generate the lock file when you upgrade things.

https://caremad.io/posts/2013/07/setup-vs-requirement/ covers it pretty well. Or https://medium.com/@sdboyer/so-you-want-to-write-a-package-m... for a fantastic read and a LOT more context.

The question is... How is this better than running 'pip freeze > requirements.lock' whenever you decide to make a new library (or version) official?

  Grouping of sub-dependency groups (e.g. a testing group).

That seems like an improvement over having multiple requirement files. Then again, I'm perfectly happy with separate dev/testing files.

One use case I don't see covered is grouping by python versions. Sometimes "soft" dependencies are only available/supported/not-broken on certain python versions.

It'd be nice to be able to mark a dependency as being conditional based on some expression, e.g. `$PYTHON_VERSION_MAJOR >= 3`.

BTW, you can do that already with requirements.txt. See https://stackoverflow.com/questions/19559247/requirements-tx...

Give it time. Moving to a native python DSL will make extending the dependency system more straightforward. Also, if they continue copying the best of Ruby's bundler they will have a way to specify python version/platform specific dependencies.

Reply on news.ycombinator.com