Ask HN: Why is the Python environment so fragile, and why do we tolerate this?

Also: the dependency solver in pip is not correct. Instead of analyzing dependencies collectively it starts downloading one dependency, then adds its dependency graph and so forth. It works most of the time but it can be stuck in a place where a solution is possible but it can't find it.

Still, pip handles simple projects well, but when you add enough dependencies that have a complicated relationship it will break down in a maddening way.

Poetry's solver is much better than pip, I don't think I've ever seen it fail to solve an satisfiable problem but I don't believe it is perfect or efficient.

The correct way to solve it is download the dependencies of the dependencies before you build them. You can download the dependencies of a wheel from pypi without downloading the whole file because wheels are ZIP files and ZIP files allow random access.

The ideal product does a search process over the metadata before it downloads the packages for real as a

https://en.wikipedia.org/wiki/Satisfiability_modulo_theories

problem. It's possible to do for wheels but for eggs it is not straightforward because you have to run the setup.py to get the dependencies. eggs are not so common these days so a disciplined operation could have a process to build their own wheels from the eggs and put them in a repository.

The ideal system also has a facility to substitute one package for another, for instance "intel-numpy" for "numpy".