Hacker News Reader

Comment by WillDaSilva | Hacker News Reader

This is a good overview of something that took me an annoyingly long time to learn. My personal preference is to keep things simple with pyenv, venv, and pip.

Tangentially related is the tool tox [1], which is often used to run a test suite inside of virtual environments created by venv, on multiple versions of Python managed by pyenv.

Now if only setuptools could work well without hackery...

[1]: https://tox.readthedocs.io/en/latest/

View on HN

Poetry does a good job of managing/publishing a package without having to custom-code a setup.py. I have a blog post about it but I don't want to spam. Poetry works well for a package of average complexity, and it's configured entirely in pypyroject.toml. I don't have experience publishing something complicated with it, though.

Wouldn’t you still need something like pip-tools to lock down subdependencies and handle conflicts?

Plain old pip and venv can do that. just "pip freeze >requirements.txt" and elsewhere "pip install -r requirements.txt", inside venvs.

I think that will end up installing the subdependency version of whatever is last in the requirements.txt. You need a dependency resolver to deal with problems with conflicting versions.

More details here: https://medium.com/knerd/the-nine-circles-of-python-dependen...

pip handles the simple cases: if you install a new pkgA that depends on 'pkgB<3', it installs the latest appropriate version of that, e.g. 'pkgB==2.5.6'. This works even if you already installed 'pkgB==3.0.2', it will uninstall that first. The problem is if some 'pkgC' depends on 'pkgB>=3'. You probably want for pip (or similar) to figure out that an older version of 'pkgC' is compatible with 'pkgB>2'.

But I actually don't want it to be too smart. Better to keep your dependencies minimal and explicit, and manually specify older 'pkgC' if you need to. I have a few non-trivial services in production, the most complex one with 16 total dependencies + sub-dependencies. That is quite manageable.

So, I strongly recommend manually curating the most appropriate versions of the few tastefully chosen dependencies you really need. Then, pip+venv can easily reproduce that exact set of dependencies anytime. I also do something very similar to this with C applications, and Go. Sub-dependencies should be a big factor in how you choose your direct dependencies.

The problem with doing things this way is that you’re not going to know there’s a problem until there’s an issue in your tests (hopefully) or production. You’ll eventually install something new, it’ll update some subdependencies to a version that another library doesn’t support, and then things get broken. Pip-tools is easy to use and it tells you there’s a problem before it’s too late.

Reply on news.ycombinator.com