Self-contained Python scripts with uv
blog.dusktreader.dev
blog.dusktreader.dev
Using comments for linters and developer notes is perfectly acceptable. However, for configuration or execution-related data, a far superior pattern would be something like:
UV_ENV = {
"dependencies": { "requests": "2.32.3", "pandas": "2.2.3" }
}
This approach has clear advantages:- It's valid Python syntax.
- It utilizes standard, easily-parsable data structures rather than ad-hoc comment parsing. It makes creation and validation smooth.
- Crucially, it adheres to a core principle: if you remove all comments from your code, it should still execute identically.
Problem is that uv probably does not want to execute anything to find out dependencies, so it would have to be a very restrictive subset of python syntax.
The fact that is is needed at all of course highlights a weakness in the language. The import statements themselves should be able to convey all information about dependencies
What languages convey the version of the dependencies in a script’s import statements?
[1] https://peps.python.org/pep-0723/
[2] https://packaging.python.org/en/latest/specifications/inline...
You’re using a magic constant that doesn’t do anything at runtime. It’s only there to be parsed by static analysis. In your case that’s uv doing the parsing but another tool might delete it as unused code. In the sense that it’s one thing pretending to be another, for me, it’s in the same category as a magic comment.
Instead, why not make a call to uv telling it what to do?:
import uv
uv.exec(
dependencies=[“clown”],
python=“>=3.10”,
)
from clown import nose
The first call can be with any old python runtime capable of locating this hypothetical uv package. The uv package sets up the venv and python runtime and re-exec(3)s with some kind of flag, say, an environment variable.In the second runtime uv.exec is a noop because it detects the flag.
https://peps.python.org/pep-0723/#why-not-use-possibly-restr...
My counterpoints to the PEP’s arguments are (1) we’re about to run Python so we presumably have a Python parser on hand anyway; and (2) for the foreseeable future it is going to be capable of parsing all previous versions of Python.
It’s a bit fast and loose though. I can see though that it’s helpful for long term stability to have two completely separate languages for dependencies versus the actual code, with the former being far more reduced and conservative than the latter.
If you use Python4.98 triple walrus operators to say requires_version:::=“>=4.98” it would definitely be annoying for any version prior to that to not even be able to parse the requirements, let alone try to meet them.
I agree that theoretically your proposed way of doing things would be conceptionally among the cleanest, but on the other hand in all kind of scripts the shebang was sort of a comment with big implications as well, so I am not sure if being dogmatic is worth it here.
It might be specified that I needs to be proper JSON. And a proper JSON is much more maintainable (and extendible) than impromptu syntax (that first starts manageable, but step by stem moves into parsing hell).
That's probably the goal. It's only there for one tool. If it's not used, we want it to have no impact on the running app. Like comments.
One important aspect to remember is that this isn't intended to be a uv specific feature. It's a (proposed) python standard feature that in the future other python package managers will implement. So whatever solution they come up with it has to work with any standard compliant package manager, not just uv.
For example if you run this on python3.6:
print("hello")
match 123:
case _: pass
you won't even get a "hello".The trick in the featured article would allow me to drop a script written in modern python syntax on my outdated ubuntu LTS box and have it "just work", while GP's suggestion would not.
Remember the specification to indicate dependencies in a comment on a script is a PEP (723) and it’s tool-agnostic.
Literally cannot get simpler than that. Making uv an importable means assuming Python is present or easily installed on every system, which if it were the case then uv wouldn't be becoming a thing.
It’s also worth noting that using comments is exactly how the shebang line works in the first place. It’s just so well-ingrained after 45 years that people don’t notice that it’s a shell comment.
It still -does- execute identically. Provided you install the same dependencies.
I don't see this as changing the semantics of the code itself, rather just changing the environment in which the code runs. In that respect it is no different from a `#!/bin/bash` comment at the top of a shell script.
https://news.ycombinator.com/item?id=43500124
https://news.ycombinator.com/item?id=42463975
I like uv and all, but I take exception to the "self-contained" claim in two regards:
1) The script requires uv to already be installed. Arguably you could make it a shell script that checks if uv is already installed and then installs it via curlpipe if not... but that's quite a bit of extra boilerplate and the curlpipe pattern is already pretty gross on its own.
2) Auto-creating a venv somewhere in your home directory is not really self-contained. If you run the script as a one-off and then delete it, that venv is still there, taking up space. I can't find any assertion in the uv docs that these temporary virtual environments are ever automatically cleaned up.
That’s a good point. I wonder if at least they are reused when you run the script several times.
It's not that the inline requirements make a new `.venv` directory or something, uv seems to link the packages to a central location and reuse them if already there.
This works but doesn’t cache anything so the download for every run is awkward. This can probably be fixed with a volume tho?
Something like this: https://hugojosefson.github.io/docker-shebang/#python
Another thing is that inline script metadata is a Python standard. When there is no uv on the system and uv isn't packaged but you have the right version of Python for the script, you can run the script with pipx: https://pipx.pypa.io/stable/examples/#pipx-run-examples. pipx is much more widely packaged: https://repology.org/project/pipx/versions.
Why is signing key with .deb/.rpm better than `curl | sh` from a HTTPS link on a domain owned by the author? .deb/.rpm also contain arbitrary shell commands.
To me, fully self-contained is something more like an AppImage
#! nix-shell -i python3 -p "python312.withPackages (pkgs: [ pkgs.boto3 pkgs.click ])"
With this, the only requirement is Nix on the system, you don't even need Python to be installed!Note that this is exactly the case in TFA - uv takes care of installing Python ad-hoc.
For those who want a really self-contained Python script, I'd like to point out the Nuitka compiler [0]. I've been using it in production for my gRPC services with no issues whatsoever - just "nuitka --onefile run.py" and that's it. It Just Werks. And since it's a compiler, the resulting binary is even faster than the original Python program would be if it were bundled via Pyinstaller.
The author's GitHub page [1] contains the following text:
Other than software development, my passion would be no
other. It's my life mission to create the best Python
Compiler I can possibly do or die trying, ... of old
age.
[0] https://nuitka.net/I could always do `uv run --with whatever-it-is-i-need hx script.py`, but that's starting to get redundant.
$ cat ~/.local/bin/uve
#!/bin/bash
temp=$(mktemp)
uv export --script $1 --no-hashes > $temp
uv run --with-requirements $temp vim $1
unlink $temp
Hope editor could support the `uv python find --script` soon. temp=$(mktemp)
trap 'unlink $temp' EXIT
# Do thingsI peeked in uv's contributing guide and issues and didn't see any CLA. In PyTorch the CLA was mentioned at the top of the contributing guide.
Although, there should have been a community fork of the last FOSS version of Anaconda. That's what happened with Redis, and Redis uses a CLA: https://github.com/redis/redis/blob/unstable/CONTRIBUTING.md...
Don't ever sign a CLA, kids. Hell, only contribute to copyleft projects. We get paid too much to work for free.
They can always stop developing or fork to a different license and all future work belongs under that license, but you can't back date licenses, so what exists is guaranteed Open Source. If you're super worried, you can create a fork and just keep it in sync.
But this is essentially true about any other OSS project so I wouldn't be concerned. As far as I'm aware, conda was never open sourced and had always distributed binaries.
[0] https://github.com/astral-sh/uv?tab=readme-ov-file#license
Because it uses PyPI I'm happy to use it as a package manager and dev tool. The worst that can happen is I have to switch back to pip etc. But I wouldn't use it as package runtime dependency. Use pyinstaller for that.
The use case for this kind of trick I think is developer utility scripts in repos. I wouldn't want to tie any of my personal utils to uv. If it needs dependencies I'll just make a package, which is dead easy now.
uv just uses pypi, so it would be just a question of changing from uv to pip, poetry or whatever, all packages would still be coming from the same place.
[1] https://bundler.io/guides/bundler_in_a_single_file_ruby_scri...
You could do this during an installation step, for example.
[1]: https://learn.microsoft.com/en-us/windows/win32/shell/fa-fil...
$> uv init --script <script_name>.py
$> uv add --script <script_name>.py <pkg1> <pkg2> ...
$> uv add --script <script_name>.py --dev <dev_pkg1> <dev_pkg2> ...
$> uv run <script_name>.py
Hope this helps :)
> uv -V
uv 0.6.10
> uv add --script foo.py --dev ruff
error: the argument '--script <SCRIPT>' cannot be used with '--dev'
Usage: uv add --script <SCRIPT> --link-mode <LINK_MODE> <PACKAGES|--requirements <REQUIREMENTS>>
For more information, try '--help'.
There is currently no mention of `uv add --script foo.py --dev ...` in https://docs.astral.sh/uv/guides/scripts/.
Inline script metadata in Python doesn't standardize development dependencies.I wrote a recent comment about how I develop scripts with `pyproject.toml` to have a regular development environment: https://news.ycombinator.com/item?id=43503171.
This was covered in a blog post about this same topic that was posted here a few days ago. According to that you have to omit the -S: https://thisdavej.com/share-python-scripts-like-a-pro-uv-and...
https://news.ycombinator.com/item?id=43500124
I haven't tried it myself, I simply changed the file association so all .py files are opened with uv run as standard.
https://docs.python.org/3/using/windows.html#python-launcher...
I think it should be something like this:
ftype Python.File=C:\Path\to\uv.exe run %L %*
If you don't use the CPython installer the Python.File file type might not be defined, so you might need to set that with `assoc` first: assoc .py=Python.FileThis is the first time that Python didn't come with "batteries included" from my POV.
Now I also have two Python dependency managers in my system. I know there are volumes to talk about Python dependency management but all these years, as long as a project had a requirements.txt, I managed to stick to vanilla pip+venv.
Which is worse than just having a default way for including metadata that's not used. That's what makes it metadata after all. Otherwise it would just be Python syntax
- Uv's killer feature is making ad-hoc environments easy (valatka.dev): https://news.ycombinator.com/item?id=42676432
- Using uv as your shebang line (akrabat.com): https://news.ycombinator.com/item?id=42855258
I also wondered why virtual environments were invented for Python when general environment managers (like Modules) already existed.
These packaging and environment problems have never been specific to Python
uv is an attempt to fix the fragments Python development environment tooling story.
That code is bad for several reasons including not catching+handling exceptions (and possibly retrying), and accessing the JSON properties w/o get()
The overhead of re-installing stuff and setting up a header seem very unnecessary to run a simple script
If this is about sending some Python for somebody else to run easily - the recipient should always check the code. You should never run arbitrary code. For example, there have been hacks performed using YAML loader (crypto exchange).
For dependencies, use the standard pyproject.toml
my approach is to use python build-in venv https://gist.github.com/Szpadel/43794d606d9924e7fea3e63fb800...
that way you can run scripts with external packages with only basic python installation
It's also a lot easier to install uv than to manage Python installations (there's a reason pyenv and the like exist).
But probably if you need specific python version it isn't best way
"Just use the system python" gets you right back to the start (oh no! It didn't parse because it used python 3.11 features and I'm still on 3.5)
This is actually the root of all Python problems, by it believing it has a right to be a core part of the operating system all the package design choices treat their installation like they're the only thing running on the machine when the realities of Python packages are they all rely on very specific versioning from the interpreter to between the packages so the idea of having a canonical version of pytorch that all your projects run on just doesn't exist.