Hacker News Reader

Comment by slightwinder | Hacker News Reader

> Save that as script.py and you can use "uv run script.py" to run it with the specified dependencies,

Be aware that uv will create a full copy of that environment for each script by default. Depending on your number of scripts, this could become wasteful really fast. There is a flag "--link-mode symlink" which will link the dependencies from the cache. I'm not sure why this isn't the default, or which disadvantages this has, but so far it's working fine for me, and saved me several gigabytes of storage.

View on HN

>There is a flag "--link-mode symlink" which will link the dependencies from the cache. I'm not sure why this isn't the default

The hard-link strategy still saves disk space — although not as much (see other replies and discussion).

Possible reasons not to symlink that I can think of:

* stack traces in uncaught exceptions might be wonky (I haven't tried it)

* it presumably adds a tiny hit to imports from Python, but I find it hard to imagine anyone caring

* with the hard-linking strategy, you can purge the cache (perhaps accidentally?) without affecting existing environments

By default it will create hard links for python packages, so it won't consume any more memory (besides the small overhead of hard links).

As far as I can tell, this only applies to the wheel contents, not to the .pyc bytecode cache created by the interpreter. If you use the defaults, Python will just create per-environment copies on demand; if you precompile with `--compile-bytecode`, uv will put the results directly in the installed copy rather than caching them and hard-linking from there.

I plan on offering this kind of cached precompiled bytecode in PAPER, but I know that compiled bytecode includes absolute paths (used for displaying stack traces) that will presumably then refer to the cached copies. I'll want to test as best I can that this doesn't break anything more subtle.

Unless it can't because you happen to have mounted your user cache directory from a different volume in an attempt to debloat your hourly backups.

In that case, you use copy OR what you can can also do, if you really care about disk usage, is use symbolic links between the drives. have a .venv sym link on drive A (raid 1) point to the uv_cache_dir's venv on drive B (raid 0). I have not tested though what happens when you unmount and sync.

... your hourly backups aren't also using the same hard-linking strategy?

They’re ZFS snapshots, so no. That’s why I’m forced to keep my cache directory in a different dataset.

Ironically, if my backup scheme were using hard links, then I could simply exclude the cache directory from backup, so I’d have no reason to do that mountpoint spiel, and uv’s hard links would work normally.

... ZFS snapshots don't produce copies of the underlying file data, do they? (Yeah, I'm still a dinosaur on ext4...)

Nothing wrong with ext4! Having choices and preferences is a good thing.

You’re correct, ZFS snapshots don’t produce copies, at least not at the time they’re being created. They work a little like copy-on-write.

Reply on news.ycombinator.com