- Pin all application versions - Don't pin or set upper bounds in libraries. Lower bounds may work. - Use automation to continuously upgrade and test new versions of everything
If you just pin, you fall behind and eventually it becomes expensive to catch up. If you don't pin, you lose repeatability. If you don't automate, the upgrade work doesn't happen reliably.
It’s rarely a pain in the ass. When it is, it’s because we’ve chosen dependencies poorly and that’s a good signal to move on to something else. Occasionally major versions require a little more time and effort, but just eat the cost and do it.
What’s painful is updating rarely and calcifying. Do the hard thing often and it stops being hard.
1. Use pip-tools to generate a pinned requirements list. This is used to build artifacts like docker images, installer bundles etc. All pins are upgraded periodically (manually). Any version changes are inspected (this requires "knowing" your dependencies a bit and which ones might cause problems). Finally the resulting artifacts are manually tested.
2. An automated build of master every 24 hours from completely unpinned versions. That is installing the package fresh from pyproject.toml. The automated test suite will reveal any upcoming breakage that would happen if we continue without upper bounds on certain packages. If there is breakage we decide whether to accommodate the new breaking dependencies or constrain them with an upper bound (perhaps recording a tech debt).