Boring Python: Dependency Management

b-list.org

2 pointsjefurii2 comments

I think poetry is a big step forward. A few years back I worked at a place where our projects were so complex that "pip install" would not be able to reliably solve dependencies -- it was by no means a "boring" choice!

pip's strategy for solving dependencies is just a tactic: it starts installing one package at a time, installs the dependencies of that package, etc. It can get into a situation where another package has dependencies that conflict with what has already been installed and pip gets stuck.

Alternately it is possible to solve the dependencies as a whole and then install everything. I wrote something that was able to gather up all the wheels necessary (and make wheels if they didn't already exist) and then the installer would just unpack the wheels.

> pip's strategy for solving dependencies is just a tactic: it starts installing one package at a time, installs the dependencies of that package, etc. It can get into a situation where another package has dependencies that conflict with what has already been installed and pip gets stuck.

I faced that issue this week. I was transferring my blog from my old computer to my new laptop. It uses Pelican and virtualenv. The 'main packages' are in a requirements.txt file (pelican, markdown, jinja2, etc). There are only a few packages in the requirements.txt. Then, I tried to run the blog on the new laptop and, surprise surprise, it did not work. One of the packages depends on another one that introduced a conflict. The solution was to run `pip3 freeze > requirements.txt` on my old laptop to get the exact version of all packages. After that, I can run the blog on my new machine. It seems obvious but I used to include only a few packages in the requirements.txt, not all of them. Maybe this approach is boring enough.

Reply on news.ycombinator.com