Hacker News Reader

Comment by mbyio | Hacker News Reader

It is a solved problem for other languages. Pip has soooo many issues.

The biggest IMO is pip doesn't provide a way to pin indirect dependencies, so there is no good way to ensure that other people working on a project are also using the same dependency versions. You can do it with extra tooling of course - and that's what poetry and pipenv do.

View on HN

Even worse-- there's no simple way for pip to print out the full dependency tree (including transitive dependencies). This makes it a nightmare when it complains about some transitive dependency version conflict and you have no idea what's causing it to be pulled in

Isn’t this what pip freeze is for?

pip freeze only pins the version of direct, not transitive dependencies. Meaning you don't get deterministic builds.

What do you mean? If you run pip freeze inside a venv it will print every user-installed module and their dependencies. When installing them in a new venv as a requirements file, you end up with the exact same set of modules.

Agree. Works fine for me exactly as you describe. I don't understand why people furiously want more than this for the common case. (Choose python version, create .venv/ directory, install packages, use installed .venv/ and do work.)

What happens when you use a different version of the same library that a dependency uses?

Sometimes it works, sometimes it doesn't. This is indeed a problem, but one for which there isn't an elegant solution.

When multiple versions of the same library are allowed to coexist in different dependency trees there is less incentive to solve these conflicts and versions proliferate (see npm).

I'd much rather solve the occasional conflict (usually solved either by settling on some older version of both parent dependencies - and later advancing as possible - or just trying to fix it and send a patch upstream).

Issues start occurring when you, for example, delete dependencies because with pip freeze you cannot ensure that you have deleted their dependencies also. The most common solution to this (that poetry and pipenv use) is to provide a lockfile to track transitive dependencies and their versions, without that (except for manually curating your dependencies) you can't ensure that you get a reproducible environment.

OK, but it does manage the versions of transitive dependencies, and there's nothing in that process stopping deterministic builds.

Adding/removing top level dependencies over time does require the use of two files (the top level requirements and the frozen/locked requirements which lists everything). Or you can list the top level requirements in setup.py and let requirements.txt be the lockfile. It would be nice if pip managed this lockfile automatically, but I'm not really interested in adding any of these newer tools to my toolchain just to manage a lockfile.

There are many packaging and distribution frustrations in Python, I don't think pip's management of dependency lists is one of them

Not so. I use pip freeze to manage my dependencies, including all transitive dependencies, and my builds are deterministic.

How do you include checksums in your freeze to catch when the package changes on the pypi server using only pip?

I don’t. PyPI no longer allows reuploading the same release.

I thought `pip freeze > requirements.txt` would give you a list of all dependecies, even indirect ones. Am I missing something?

I just so wish that Python tooling would be better, in large part pip is at fault here. If there was no conda I probably wouldn't even touch Python as things look very bleak especially if you need to run anything across different operating systems.

Reply on news.ycombinator.com