Hacker News Reader

Comment by heisig | Hacker News Reader

I recently switched to uv, and I cannot praise it enough. With uv, the Python ecosystem finally feels mature and polished rather than like a collection of brittle hacks.

Kudos to the uv developers for creating such an amazing piece of software!

View on HN

Yeah, switched to writing python professionally ~4 years ago, and been low key hating the ecosystem. From a java and javascript background, it's mostly been npm/mvn install and it "just works". With python, there's always someone being onboarded that can't get it to work. So many small issues. Have to have the correct version per project, then have to get the venv running. And then installing it needs to build stuff because there's no wheel, so need to set up a complete c++ and rust toolchain etc., just to pull a small project and run it.

uv doesn't solve all this, but it's reduced the amount of ways things can go wrong by a lot. And it being fast means that the feedback-loop is much quicker.

I cannot share the same experiences. mvn is a buggy mess, randomly forgetting dependencies, and constantly needing a full clean to not die on itself. npm and the entire js ecosystem feels so immature with constant breaking changes, and circular dependency hell, when trying to uppgrade stuff.

That's an issue with the packages themselves though, not with package management as a whole. You and the comment above you are talking about different things. While there's plenty of pain to be had with npm, if you have a project that used to work years ago, you can generally just clone, install and be done, even if on older versions. On Python this used to mean a lot of hurt, often even if it was a fresh project that you just wanted to share with a colleague.

For value of "years" greater than 1?

Node/NPM was a poster child of an ecosystem where projects break three times a week, due to having too many transitive dependencies that are being updated too often.

Projects breaking so frequently on npm and node is simply not the case unless you are trying upgrade an old project, one dependency per day…

This argument makes no sense. Your dependencies don't change unless you change them, npm doesn't magically update things underneath you. Things can break when you try to update one thing or another, yes, but if you just take an old project and try and run it, it will work.

Assuming the downloads still exist? Does NPM cache all versions it ever distributed?

That's always one major thing I saw breaking old builds: old binaries stop being hosted, forcing you to rebuild them from old source, which no longer builds under current toolchains - making you either downgrade the toolchain that itself may be tricky to set up, or upgrade the library, which starts a cascade of dependency upgrades.

It's not like Node projects are distributed with their deps vendored; there's too much stuff in node_modules.

You're using a different, not hosted anymore package, three times a week? That's somewhere between very unusual and downright absurd.

Yes you can find edge cases with problems. Using this as an argument for "breaks 3 times per week" does not hold.

No, I was using this as an argument for why I don't expect Node projects older than a year or two to be buildable without significant hassle.

(Also note that outside the web/mobile space, projects that weren't updated in a year are still young, not old. "Old" is more like 5+ years.)

The two things are related. If your typical project has a dependency DAG of 1000+ projects, a bug or CVE fix somewhere will typically cause a cascade of potentially breaking updates to play out over multiple days, before everything stabilizes. This creates pressure for everyone to always stay on the bleeding edge; with a version churn like this, there's only so many old (in the calendar sense) package dists that people are willing to cache.

This used to be a common experience some years back. Like many others, I gave up on the ecosystem because of the extreme fragility of it. If it's not like that anymore, I'd love to be corrected.

I don't know if it is still as fragile as you remember but if you just never update your package-lock then it is super stable as you (transitive) dependencies never change.

The non-trivial exception being if some dependecy was downloading resources on the fly (maybe like a browser compat list) or calling system libraries (eg running shell commands)

> Does NPM cache all versions it ever distributed?

Yes it does, that's the whole point. You can still go and install the first version of express ever put on npm from 12 years ago. You can also install any of the 282 releases of it that have ever been put on npm since then. That's the whole point of a registry, it wouldn't be useful if things just disappeared at some random point in time.

The only packages that get removed are malware and such, and packages which the vendor themselves manually unpublish [0]. The latter has a bunch of rules to ensure packages that are actually used don't get removed, please see the link below.

[0] https://docs.npmjs.com/policies/unpublish

IIRC there is a package whose whole point is to include everything else in its package.json and make them ineligible for unpublish.

> npm doesn't magically update things underneath you

It used to prior to npm 5 when lockfiles were introduced (yarn introduced lockfiles earlier).

I've seen mvn projects that spin like a top and others that were a disaster.

I think it's little recognized that there is a scaling limit for snapshots. If you have 20 people developing 20 projects and they are co-located in the same room with the server builds work 50-80% of the time and people think it's fine. If you're the one guy who is remote and has a slow connection, builds work 0% of the time. The problem is that at slightly different times you get slightly different snapshots that aren't compatible with each other -- it's a scaling problem because if you add enough developers and enough projects it will eventually get you.

I've worked at other places when the mvn clean was necessary every time; other developers thought this shouldn't be necessary and I was a doofus except I was able to make consistent progress like a ratchet on the project and get it done and they weren't.

Where I am now mvn is just fine, whenever it screws up there's a rational explanation and we're doing it wrong.

I'm not saying mvn or npm is perfect. But the issues they have are consistent. My coworker and I would either have the same issues or not any issues. But with python it's probably more ways of running the project in the team than there are people, all with small tweaks to get it working on their system.

Python has been mostly working okay for me since I switched to Poetry. (“Mostly” because I think I’ve run into some weird issue once but I’ve tried to recall what it was and I just can’t.)

uv felt a bit immature at the time, but sounds like it’s way better now. I really want to try it out... but Poetry just works, so I don’t really have an incentive to switch just yet. (Though I’ve switched from FlakeHeaven or something to Ruff and the difference was heaven and hell! Pun in’tended.)

A lot of Wagtail usage is with Poetry. Tends to be projects with 30-50 dependencies. It "just works" but we see a lot of people struggle with common tasks ("how do I upgrade this package"), and complain about how slow it is. I don’t have big insights outside of Wagtail users but I don’t think it’s too different.

n=1 but i've tried "manual" .venv, conda/miniconda, pipenv, poetry, and finally now at uv. uv is great. poetry feels like it's focused on people who are publishing packages. uv is great to use for personal dev, spinning up/down lots of venv, speedy, and uvx/uv script is very convenient compared to having all my sandbox project in one poetry env.

Ok, you convinced me to give it a try. Tbh, I am a casual user of python and I don't want to touch it unless I have a damn good reason to use it.

You do not need a damn good reason for this. Just try it out on a simple hello world. Then try it out on a project already using poetry for eg.

uv init

uv sync

and you're done

I'd say if you do not run into the pitfalls of a large python codebase with hundreds of dependencies, you'll not get the bigger argument people are talking about.

I don't think you need to sync, do you? It always just does it when running.

That said, I do wish uv had `uv activate`. I like just working in the virtualenv without having to `uv run` everything.

I do usually include instructions in our READMEs to do a `uv sync` as install command, in order to separate error causes, and also to allow for bootstrapping the venv so that it's available for IDEs.

That makes sense, thanks.

You can still `source .venv/bin/activate(.fish)` and skip the uv run bit. I have Fish shell configured to automatically activate a .venv if it finds one in a directory I switch to.

I do do that, can you please share your fish script to autoload it? I have something for Poetry envs, but not venv dirs.

Sure thing - so I mostly ended up using this for activating a .venv in a fabfile directory using this...

    function __auto_fab --on-variable PWD
        iterm2_print_user_vars
        if [ -d "fabfile" ]
            if [ -d "fabfile/.venv" ]
                if not set -q done_fab
                    and not set -q VIRTUAL_ENV
                    echo -n "Starting fabfile venv... "
                    pushd fabfile > /dev/null
                    source .venv/bin/activate.fish  --prompt="[fab]"
                    popd > /dev/null
                    set -g done_fab 1
                    echo -e "\r Fabfile venv activated         "
                end
            else
                echo "Run gofab to create the .venv"
            end
        end
    end

I've since deleted the one to do a .venv in this directory, but I think it was roughly this...

    function __auto_venv --on-variable PWD
        if [ -d ".venv" ]
            if not set -q done_venv
                echo -n "Starting venv... "
                source .venv/bin/activate.fish  --prompt="[venv]"
                set -g done_venv 1
                echo -e "\r Venv activated         "
            end
        end
    end

(just tested that and it seems to work - the --prompt actually gets overridden by the project name from uv's pyproject.toml now though so that's not really necessary, was useful at some point in the past)

These live in ~/.config/fish/conf.d/events.fish

Thank you!

I'm not them, but I use `direnv` for this. Their wiki includes two layout_uv[1] scripts, one that uses `uv` just to activate a regular venv and a second that uses it to manage the whole project. I use the latter.

[1] https://github.com/direnv/direnv/wiki/Python

That's great, thanks! I use direnv but didn't know they had this.

Custom layouts are awesome. You can set up any script to run when direnv runs, so you can support just about anything you want even before direnv adds a builtin.

I keep going back and forth on ‘uv run’. I like being explicit with the tooling, but it feels like extra unneeded verbosity when you could just interact with the venv directly. Especially since I ported a bunch of scripts from ‘poetry run’

> I am a casual user of python and I don't want to touch it unless I have a damn good reason to use it.

I... what? Python is a beautiful way to evolve beyond the troglodyte world of sh for system scripts. You are seriously missing out by being so pertinently against it.

Just you wait till someone shows you how Rust is to Python what Python is to shell scripts. For one, null safety is a major issue in most corporate Python code, and much less of an issue in Rust code.

Rust is decidedly not a scripting language.

Don't get me wrong, Rust is great and I use it too, but for very different purposes than (system) scripts.

Now, if I hadn't read literally the same message for Pipenv/Pipfile and poetry before, too...

Python is going through package managers like JS goes through trends like classes-everywhere, hooks, signals etc

There have been incremental evolutionary improvements that were brought forth by each of the packages you named. uv just goes a lot further than the previous one. There have been others that deserve an honorary mention, e.g. pip-tools, pdm, hatch, etc. It's going to be very hard for anything to top uv.

But how does it work with components that require libraries written in C?

And what if there are no binaries yet for my architecture, will it compile them, including all the dependencies written in C?

IMO if you require libraries in other languages then a pure python package manager like uv, pip, poetry, whatever, is simply the wrong tool for the job. There is _some_ support for this through wheels, and I'd expect uv to support them just as much as pip does, but they feel like a hack to me.

Instead there is pixi, which is similar in concept to uv but for the conda-forge packaging ecosystem. Nix and guix are also language-agnostic package managers that can do the job.

But for example, if I install the Python package "shapely", it will need a C package named GEOS as a shared library. How do I ensure that the version of GEOS on my system is the one shapely wants? By trial and error? And how does that work with environments, where I have different versions of packages in different places? It sounds a bit messy to me, compared to a solution where everything is managed by a single package manager.

You could use a package manager that packages C, C++, Fortran and Python packages, such as Spack: here's the py-shapely recipe [1] and here is geos [2]. Probably nix does similar.

[1]: https://github.com/spack/spack/blob/develop/var/spack/repos/... [2]: https://github.com/spack/spack/blob/develop/var/spack/repos/...

That's what I mean, in this case pip, uv, etc. are the wrong tool to use. You could e.g. use pixi and install all python and non-python dependencies through that, the conda-forge package of shapely will pull in geos as a dependency. Pixi also interoperates with uv as a library to be able to combine PyPI and conda-forge packages using one tool.

But conda-forge packages (just like PyPI packages, or anything that does install-time dependency resolution really) are untestable by design, so if you care for reliably tested packages you can take a look at nix or guix and install everything through that. The tradeoff with those is that they usually have less libraries available, and often only in one version (since every version has to be tested with every possible version of its dependencies, including transitive ones and the interpreter).

All of these tools have a concept similar to environments, so you can get the right version of GEOS for each of your projects.

Indeed, I'd want something where I have more control over how the binaries are built. I had some segfaults with conda in the past, and couldn't find where the problem was until I rebuilt everything from scratch manually and the problems went away.

Nix/guix sound interesting. But one of my systems is an nVidia Jetson system, where I'm tied to the system's libc version (because of CUDA libraries etc.) and so building things is a bit trickier.

with uv (and pip) you can pass the --no-binary flag and it will download the source code and build all you dependencies, rather than downloading prebuilt binaries.

It should also respect any CFLAGS and LDFLAGS you set, but I haven't actually tested that with uv.

I just tried --no-binary with the torchvision package (on a Jetson system). It failed. Then I downloaded the source and it compiled without problems.

What exactly prevents you from creating your own packages if you want to use your system package manager?

On Alpine and Arch Linux? Exactly nothing.

On Debian/Ubuntu? maybe the convoluted packaging process, but that's on you for choosing those distributions.

On Nvidia/Jetson systems, Ubuntu is dictated by the vendor.

You are describing two different problems. Do you want a shapely package that runs on your system or do you want to compile shapely against the GEOS on your system. In case 1 it is up to the package maintainer to package and ship a version of GEOS that works with your OS, python version, and library version. If you look at the shapely page on pypi you'll see something like 40 packages for each version covering most popular permutations of OS, python version and architecture. If a pre-built package exists that works on your system, then uv will find and install it into your virtualenv and everything should just work. This does means you get a copy of the compiled libraries in each venv.

If you want to build shapely against your own version of GEOS, then you fall outside of what uv does. What it does in that case is download the all build tool(s) specified by shapely (setuptools and cython in this case) and then hands over control to that tool to handle the actual compiling and building of the library. It that case it is up to the creator of the library to make sure the build is correctly defined and up to you to make sure all the necessary compilers and header etc. are set up correctly.

In the first case, how does the package maintainer know which version of libc to use? It should use the one that my system uses (because I might also use other libraries that are provided by my system).

The libc version(s) to use when creating python packages is standardised and documented in a PEP, including how to name the resulting package to describe the libc version. Your local python version knows which libc version it was compiled against and reports that when trying to install a binary package. If no compatible version is found, it tries to build from source. If you are doing something 'weird' that breaks this, you can always use the --no-binary flag to force a local build from source.

This type of situation is why I use Docker for pretty much all of my projects—single package managers are frequently not enough to bootstrap an entire project, and it’s really nice to have a central record of how everything needed was actually installed. It’s so much easier to deal with getting things running on different machines, or things on a single machine that have conflicting dependencies.

Docker is good for deployment, but devcontainer is nice for development. Devcontainer uses Docker under the hood. Both are also critically important for security isolation unless one is explicitly using jails.

UV is not (yet) a build system and does not get involved with compiling code. But easily lets you plug in any build system you want. So it will let you keep using whatever system you are currently using for building your C libraries. For example I use scikit-build-core for building all of my libraries C and C++ components with cmake and it works fine with uv.

    uv build
    Building source distribution...
    running egg_info
    writing venv.egg-info/PKG-INFO
    Successfully built dist/venv-0.1.0.tar.gz
    Successfully built dist/venv-0.1.0-py3-none-any.whl

I guess it depends on what you mean by a build system. From my understanding uv build basically just bundles up all the source code it finds, and packages it into a .whl with the correct metadata. It cannot actually do any build steps like running commands to compile or transform code or data in any way. For that you need something like setuptools or scikit-build or similar. All of which integrate seamlessly with uv.

I see what you mean. You can use it with mise that has build support.

It actually does exactly what pip does depending on your configured build backend, so if you have your pyproject.toml/setup.py configured to build external modules, `uv build` will run that and build a binary wheel

Yes, that's my point. You need to bring your own 'real' build system to make uv doing anything non-trivial. And the fact that this work transparently with uv is a very good thing.

Yes it'll build any dependency that has no binary wheels (or you explicitly pass --no-binary) as long as said package supports it (i.e. via setup.py/pyproject.toml build-backend). Basically, just like pip would

What about using https://github.com/RustPython/RustPython also?

Unlike uv this tool is unlikely to solve problems for the average Python user and most likely will create new ones.

Agree, however for user who want to get faster speed out of python wouldn't that just work with rustpython? It can also run in the browser then.

RustPython is just an interpreter written in Rust. There's no reason why it would be meaningfully faster than CPython just because it's written in Rust rather than C. Rust adds memory safety, not necessarily speed.

A new and immature interpreter is going to have other problems:

- Lack of compatibility with CPython - Not up to date with latest version features - Incompatibility with CPython extensions

RustPython is a cool project, but it's not reached the big time yet.

Reply on news.ycombinator.com