Thoughts on upper bounds for Python dependencies – conda-forge vs. PyPI

pip doesn't correctly solve dependencies in all situations, that's a problem that needs to be fixed before anything else. The last thing we need is another almost-right-but-really-broken solution like conda. People are coming to the conclusion that "python sucks" and moving on to other languages because the python community has refused to look objectively at the package management situation.

Completely successful package management systems like npm and mvn separate dependency metadata from the rest of the package so the package manager can download parts of the dependency graph and find a solution before it downloads any packages. Pip and poetry on the other hand fail because to install eggs they can have to run the setup.py (run arbitrary Python code!) just to know other dependencies. Pip in particular just starts downloading and installing packages optimistically... If it is lucky it "just works", if it isn't some combination of packages it installed are incompatible with the next one you want to install and you're stuck.

If you give up on eggs I think you can make a package manager that works like maven because you can get the deps out of a wheel with at most three http range requests. Eggs are close to extinct, although a sophisticated user who needs an egg or two for a project could build a wheel from it and point the package manager at it.

On one level I like the idea of looking up what symbols a package uses and this might be good for an analysis tool, but for a package management tool that you're going to run over, over, and over again I think it the road to make something worse than pip, not better.

(One thing I'd like to see that would further make conda obsolete is some ability to override dependencies, that is, say that "numpy-mkl" is a substitute for "numpy" and with such a capability people could tell the system to extend an upper bound from what it would be otherwise.)