Another obligatory plug for Nix, which officially supports Linux and macOS, and seems to work on Windows (Cygwin, WSL), BSD, GNU/HURD, etc. too. It can run system-wide, or limited to an unprivileged user's home dir.
Nix markets itself as a "purely functional package manager", but after using it for years I dislike this characterisation. I think of Nix as a build tool, like Make, but scalable & composable enough to "build" an entire OS (or fleet of cloud services, if you use NixOps).
In this context, we can build Python interpreters with our desired packages 'baked in' to their site-packages dir:
rec {
nixpkgs = import <nixpkgs> {};
my-scientific-python = nixpkgs.python3.withPackages (pyPkgs: [
pyPkgs.matplotlib pyPkgs.numpy
]);
my-game-dev-python = nixpkgs.python3.withPackages (pyPkgs: [ pyPkgs.pygame ]);
}
These are similar to venvs, but they're immutable and created atomically.
We can build our own bin/ directories with arbitrary contents, e.g.
nixpkgs.buildEnv {
name = "my-binaries";
paths = [
nixpkgs.gcc
nixpkgs.valgrind
nixpkgs.shellcheck
my-scientific-python
];
}
This results in a folder containing a bin/gcc, bin/valgrind, bin/shellcheck, bin/python3, etc. where that python3 binary has our desired packages available.
If we want to make an "application", e.g. to "install" on our system, we don't need to expose these underlying details; e.g. there's no need for any 'global consistency'. For example, we can write scripts which "bake in" references to their dependencies, e.g.:
writeScript "hello" ''
#!${my-game-dev-python}/bin/python3
import pygame # This is guaranteed to be available, thanks to our shebang
# We can splice in references to any file/directory we like
import json
with open("${./my-config.json}", "r") as f:
config = json.loads(f.read())
import subprocess
subprocess.run(["${nixpkgs.firefox}/bin/firefox", "http://news.ycombinator.com"])
''
When we build scripts like this, those ${} splices will be replaced by the paths to real files/directories built by other Nix builders. If we add the above script somewhere on our $PATH, e.g. called "hello", then we can run it via the command "hello", and it will start up a particular Python interpreter with Pygame installed, then launch Firefox on Hacker News; yet Python and Firefox themselves will
not be "installed" (i.e. they won't be on our PATH). Other scripts and tools are free to use their own Python interpreters if they like.
We can do the same with any language. The 'writeScript' builder used above is pretty trivial (writes the given string to a file with the given name, and sets its executable bit); we can use more complicated builders, to do any compiling, testing, etc. we like. The 'runCommand' builder is a general-purpose way to run Bash code, e.g.
nixpkgs.runCommand "my-pip-env"
{
buildInputs = [ gcc gnutar gzip python3Packages.pip ];
}
''
mkdir "$out"
pip install -t "$out" my-favourite-package
wget "https://example.com/source.tar.gz"
tar xf source.tar.gz
cp source/my-extracted-file.txt "$out/sacred.txt"
mkdir -p "$out/foo/bin"
gcc -o "$out/foo/bin/runFoo" source/foo.c
# etc.
''
Nix actually has built-in support for downloading tarballs, checking out git repos, etc. so the above is just illustrative (we should also split up these separate-looking steps into separate builders, for better caching)