Hacker News Reader

Hacker News

Comment by tbran | Hacker News Reader

Parent Full thread

tbran

This is a reminder that the excellent pyenv [0] project can help you manage all your Python versions.

- Set global and per-project python versions

- Not written in Python.

- Shims your PATH

- Linux, Mac, Windows

[0]: https://github.com/pyenv/pyenv

View on HN

as of late I've settled on using asdf[0] rather than nvm, pyvenv etc.

[0] https://asdf-vm.com/guide/getting-started.html

Agreed, asdf is the last stop for me, and uses pyenv internally anyway for python

A hidden gem of pyenv is its 'python-build' plugin, which just lets you build and install any Python version in the least number of steps:

  git clone https://github.com/pyenv/pyenv.git
  cd pyenv/plugins/python-build/bin
  ./python-build --definitions
  ./python-build 3.10.8 /opt/python/3.10.8
  PYTHON_CONFIGURE_OPTS="--enable-shared" ./python-build 3.10.8 /opt/python/3.10.8

I’ve regularly used ‘python-build’ in docker container construction to build a minimal (or customised) self contained Python binary, libs tool chain, etc, in an isolated path to make copying to from the build container to the final artefact container. It’s just all around excellent tooling.

> Pyenv does not officially support Windows and does not work in Windows outside the Windows Subsystem for Linux.

I don't only use Win but I do use windows, so having to use different tooling makes pyenv a hard to swallow pill

There is a windows port that works great! https://pyenv-win.github.io/pyenv-win/

There is a fork: https://github.com/pyenv-win/pyenv-win

`python3.10 -m venv` or `python3.11 -m venv` the default approach works fine and I don't need pip anything to get a virtualenv, what's the selling point of pyenv?

In your example, how did you install python 3.10 and 3.11? That's part of what pyenv solves for you.

Dont't most linux distros have separate packages for most recent python major versions, so that they can't installed in parallel?

For most major distros, the oldest version of Python that they ship is usually the one that's used by other packages that depend on Python. It's not uncommon - especially for libraries - to need testing on something older.

Nix has every maintained python version, so does Arch User Repository, and those can definitely be in parallel.

it's very easy to install them, probably easier than learning pyenv but I have not tried. I just want to use the default settings as much as possible, they're guaranteed to stay as long as python is alive and typically have less surprises for me on daily coding.

Whether it's easy or not depends on the OS you're working on. On Windows, it's just another executable installer, so it's trivial. On macOS, the official installers are terrible (they have no uninstaller), so you want another way. On Linux etc., some distros have only one Python version in the repos, some have two or three, but the latest versions of all reasonable distros won't let you install Python 3.4.

> probably easier than learning pyenv

Yeah “pyenv install 3.10.8” is basically rocket surgery, near impossible to learn.

Install a python version on your system:

    $ pyenv install 3.11-dev (or use pyenv install --list to show list all available versions to install)

Use it in your current shell:

    $ pyenv shell 3.11-dev

Use it in the project directory:

    $ pyenv local 3.11-dev

Use it as the global default:

    $ pyenv global 3.11-dev

All of the mentioned commands can be called with no arguments to show the current python version for the context.

There you go, now you know pyenv. Call it with no arguments to show the other possible commands.

Not really windows, WSL. There's windows fork recommended. Classic python splitting the ecosystem up. Why something like this isn't in go or rust that's actually cross platform capable just seems like excess effort.

Because thanks to WSL the lingua franca of scripting languages is bash and that’s how pyenv started. Rewriting when it works fine on Windows and every dev I know uses WSL anyway is excess effort.

Hell, the fork could have used Go or whatever too but it went with a bunch of .bat scripts.

Hello from corporate America where WSL is most certainly not standard and will require a virgin sacrifice if you want to get it approved by IT.

Which is to say, no pyenv is not an option for everyone.

Indeed, I recently worked on a fairly simple deployment for a state government agency. When I described the process to their IT staff and mentioned that we prefer using WSL, they looked at me like I had grown 2 heads.

Even aside from that, WSL is only useful if you are writing code that will run on Linux at the end of the day. This is usually true for web apps, but that's not the only thing people write in Python. Libraries, in particular, need to be able to target Windows directly.

or even if you want to build a python binary for windows, you can't cross compile it as far as I've found

I should maybe test out pyenv again. I tried it several years ago but I ran into some limitations. Or at least I thought I did, I might just have used it wrong. My main usecase, back then at least, was to have just one global-like environment for every Python version I was working with. And sometimes and additional one for debugging/testing some different packagecombinations. Back then it seemed like, to me, that pyenv only worked on a per-project basis, but I might just have followed incomplete guides and not looked into it deep enough.

How does pyenv compare with conda, mamba, pip and poetry? I typically use conda but ever since they broke with python 3.10 I have been considering moving to other environment managers.

I have not been able to understand why conda is so popular. I have no trouble with scientific computing with plain old `pyenv` and have never had any issues with C extensions or compilation with plain old `pip`.

Conda seems to be the most prone to getting in weird states or just hanging while "Solving environment." I have been happier leaving it behind.

Really the only two I would even consider using now are pyenv and poetry.

What's your platform? Issues with compilation are far more common on Windows, for example.

The other thing about Conda is that it doesn't cover just Python and Python packages, but all kinds of software that may be directly or indirectly relevant. For example, on Windows, there are Conda packages for the Windows SDK, and for various C++ compilers.

Try Pyenv for the Python version and poetry for the venv and dependency management.

Once you're used to the workflow it's pretty smooth. (We switched from conda 2 years ago)

According to StackOverflow the conda 3.10 bug was fixed this January. https://stackoverflow.com/a/70614013/733092

Is there a problem with venv? I am seriously curious. I use venv for 10 years and so far it was able to do everything I ever wanted.

There's no problem with venv. Python is just a popular language and people have built a lot of tools.

venv is great, but it requires the Python version you want already installed. This is pyenv's job.

I love pyenv but hate that it forces me to specify the patch version when installing python. Most of the time I wish I could just enter 3.9 and have it give me the latest.

> Set global and per-project python versions

Does that mean I can do things like install black and jupyter once and use that install across projects?

Yes? I mean that's just how standard python works as well.

I use anaconda and every time I create a new environment I have to install jupyter and black in order to have access to them after running `conda activate $envname`.

Or just use Conda/Mamba.

Reply on news.ycombinator.com