Ask HN: How come there is no one recommended way to setup Python environment?
If you ask this to 10 people, you can get 10 answers for that…
Cant we agree on how we can setup and start python?
Cant we agree on how we can setup and start python?
I think it may be in the class of things Python people COULD fix but WONT fix.
For me, the biggest bugbear is how often you need to work through pip not liking install frameworks like brew. It drives very strongly to telling you to "virtualise" but that means having link farms and context specific execution envirements baked into your praxis.
If (like me) you're used to just vi /tmp/file.py and hacking, the burdens of pip not wanting to break things (tm) become high. Could e.g. python not have SYSPATH:OTHERPATH:VERSION-SPECIFIC-PATH:~/.PYENV: included in it, so we can install pip stuff in homedirs and not have to worry about smashing the system install? (avalanche of "bad idea because.." coming)
Perl kind of had this. You can live with old perl libs through at most one ENV setting to point to your install path.
It's nowhere near as onerous as you make it sound.
I get the impression that you miss the experience of having a single environment that you can hack in to your heart's content, deal with package conflicts within that environment when you come to them, and stay isolated away from the system.
Here's one way to do that step by step on Debian:
$ sudo apt-get python3-venv python3-pip
$ python3 -m venv ~/Desktop/hack-venv
$ alias hack-python='~/Desktop/hack-venv/bin/python'
$ alias hack-pip='~/Desktop/hack-venv/bin/pip'
> Could e.g. python not have SYSPATH:OTHERPATH:VERSION-SPECIFIC-PATH:~/.PYENV: included in it,
That's exactly what virtual environments do. For example, on Linux, the SYSPATH:OTHERPATH part is `/usr` for the system Python, or the path to a virtual environment; the VERSION-SPECIFIC-PATH is `lib/python3.x/site-packages` (where `x` is replaced with the appropriate minor version).
The entire point of setting up a virtual environment is that you choose a separate SYSPATH:OTHERPATH part, (and it sets up the other sub-folders, depending on the Python version for that environment) so that you can have a separate folder where you put libraries specific to that environment.
> so we can install pip stuff in homedirs and not have to worry about smashing the system install?
You can install for the system install in a specific home directory, using the `--user` flag for Pip. (On newer systems that have opted into the new protection scheme, you also need `--break-system-packages` - it's named like that for a reason.) Python's startup process will add that folder to `sys.path` automatically when you run the system Python. This allows you to install those things without admin privileges (so you don't run arbitrary code as root from someone's `setup.py`)
But such packages can still "smash" the system install because of how absolute imports work - the package that you install could shadow a name of something that a system-installed library wants to `import`. That's why the new "externally-managed-environment" system exists (https://software.codidact.com/posts/291839).
If you want some other environment to use that same home directory (or another one of your choosing), you can add a `.pth` file to its `site-packages`. If you want every environment to use it, set the PYTHONPATH environment variable (and you can also use a command-line flag to disable considering PYTHONPATH). These techniques are fully documented starting from https://docs.python.org/3/library/sys_path_init.html ; you should also read the documentation for the `site` standard library module that runs by default at startup (https://docs.python.org/3/library/site.html).
(Yeah, that feels like cheating)
1. Backwards compatibility concerns. Python devs hate churn in general, and the ones responsible for major projects like Pip seem traumatized by the 2->3 migration in particular (they not only had to make their own code work, but get pieces to work together in an era before "ecosystem" was even a common buzzword, and deal with complaints from clients).
2. Everyone agrees that "there should be one way to do it", per the Zen. But there is very little agreement on what's actually included in "it", and no agreement on how those tasks should (or may) be distributed across tools. Some people advocate for all-in-one solutions, others for the Unix philosophy. (My personal conclusion is that users should have an integrated tool for every common user-facing task, while devs should add to that with individual tools for individual dev tasks.)
3. The distinction between "applications" and "libraries" in Python is subtle, but hasn't been very well addressed. People have tried for years (and are currently trying again) to figure out a standard for lock files. Pipx smooths over the gap between library and application installation requirements by holding on to its own copy of Pip and using it (through perfectly well documented, but under-appreciated command-line flags) to install into specific environments which it creates (with the standard library `venv`). But while it could function as a manager for those two tools, it artificially restricts itself not to install libraries into a venv that doesn't already contain an application.
4. Python libraries have to handle a lot of extensions written in other languages. Sometimes it's necessary to compile those on the user's machine, and people want that process to be fully automated - but that leaves the problem of how to specify build-time requirements that have nothing to do with Python (for example, a C compiler).
5. The standard tools have a bunch of random limitations and flaws, and nobody agrees on what's worth fixing or how to fix it. I could probably rant for hours about this.
6. Linux distros (especially Debian and everything downstream of it) provide Python because they want to use Python to provide essential services - not so that the user can have the experience of using Python according to how the official documentation says it's supposed to work.
If you're talking specifically from the user's side: Pip and venv are still the defaults. It's worth learning how to use them better: you can tell `venv` to exclude Pip from the new environment, and you can tell a single globally-installed Pip which environment to install into. (There won't ever be any getting around understanding the concept of an environment, because if you have countless third parties offering libraries, there's no way to assure that they'll all play nice with each other.) You can ignore the system package manager completely (except for possibly installing packages to give you Pip and venv). You can make your own wrappers to auto-activate environments on some trigger, or to detect an environment based on the PWD and/or the path to the code. But I agree that there are a lot of rough edges here, which is part of why I've been designing my own competitor (no GitHub repo for it yet; I'm hoping to have something before the end of the year, but other projects are a priority for me).
If you're talking specifically about installing Python itself - as far as python.org is concerned, the standard way on Windows and Mac is to run their installer, and the standard way on Linux is to install from source using their source tarball. Tools like Pyenv are really just wrappers around this. (I can't comment on Homebrew as it's been about 20 years since I owned a Mac.)
If you're talking about choosing between installations to run, this is also pretty standard, although the standard again has flaws. I've written about this at https://software.codidact.com/posts/291616/ .