Hacker News Reader

Comment by seamossfet | Hacker News Reader

It's crazy to me that in 2025 we still haven't figured out python dependency management

People want to be able to have a free-wheeling NodeJS-like ecosystem (in a language that shares the whole "`import`s are tracked down at runtime rather than being statically linked" thing, which in turn means you can only practically have one version of the same dependency in a given environment) while also having pieces that are, well, as complex as SciPy, Tensorflow etc. are. And they want to integrate all the missing functionality without anyone ever getting their workflow broken, because everyone still remembers the 2->3 transition. They want this even though the traditional `setup.py`-only way of doing things was abhorrently designed (i.e. you have to run arbitrary code at install time to get any metadata at all - including metadata about build dependencies, the absence of which might cause that code to fail).

Of course we haven't figured it out yet.

I love the fact that almost every single answer to your comment is a completely different take.

I didn't realize how bad it was until I recently looked into doing a Python Qt application. There's two competing Python language bindings for Qt: PyQt and PySide. The latter appears more free and active, so that seemed the way to go.

They have a compatibility matrix. It's mostly red. https://wiki.qt.io/Qt_for_Python#Python_compatibility_matrix

How do you even make your stuff dependent on/broken with specific Python versions? I mean how in hell?

The fact that venv is so widely used in Python was always an indication that not all was well dependency-wise, but it doesn't seem like it's optional anymore.

My guess would be that because it's an integration with a sprawling native library, there is a lot of code which touches Python's own C API. That is not entirely stable, and it has a lot more source stability than binary stability:

https://docs.python.org/3/c-api/stable.html

As time passes, it makes sense to support only recent versions of both Qt and Python, hence that matrix.

All I see is that newer library versions drop support for old Python versions (seems natural, unless you have unlimited resources). For example, If you need Python 2 support, use corresponding old versions.

The problem is that people are asking for more and more out of these tools. The tools we had last year are great at solving all the problems we had 20 years ago. But as long as we keep coming up with new problems, we're going to keep needing new tools.

We figured, but people can't stand still and just Do Shit (TM).

Poetry works fine and solves dependencies and peer dependencies. But I guess the javascript-style churn of 'every week a new tool which is better than the formers' has arrived too in Python land.

Poetry in theory works fine but I found it a pain to use on Windows. As in flakey. A project works, then a dependency can't be found, so I reboot and then it can. The tools to manage and diagnose why it is not installing to the right env are non existant. I am probably not holding it right... but should I need to deep dive rabbit hole into installing dependencies. It is a non issue in NPM, .NET's Nuget, Haskell Stack, Ruby, etc.

Debian had it figured out, then they had to go and ruin it all by hopping on the venv bandwagon.

Dependency management and imports (without a proper namespace solution) make me angry beyond reason. I love and hate Python.

In fairness, this is a response to a need which became common only relatively recently

Don't worry, there's still a couple of months left until then.

contrary to popular belief, Python itself doesn't have a dependency problem.

The native ABI PyObject Cuda .so .dll shit had wayy to many serious problems.

Other lang also had the same problem, think something like cgo or JNI

What do you mean? Just venv your venv.

It would be nice if Python could actually support this.

But 'venv detection' is conventionally (everywhere) done by a stupid, never-designed involving inspecting argv0. At the same time, most Python packaging tools make various assumptions like "if I'm in a venv, I can write into it", or "there will only ever be one level of venvery, so I'm not in a venv iff I'm looking at the system Python".

Nesting/layering venvs has never been supported by any Python dependency management tool except for experiments that were rejected by more senior people in the community, the maintainers of major dependency management tools, etc.

Same kind of thing goes for allowing multiple versions of a dependency in the dependency tree at runtime.

There's little appetite for fixing Python's dependency management issues in a principled or general way, because that would require breaking things or adding limitations. You can't have dependencies partially managed by arbitrary code that runs only at install time and also build good packaging tools, for instance.

What are you talking about? Everyone just uses OmegaStar.

Reply on news.ycombinator.com