Managing Python Versions with Pyenv | Hacker News Reader

Managing Python Versions with Pyenv

thepythoncorner.com

70 pointsEntICOnc110 comments

Probably going to catch hate for this, but the Python package/environment management debacle is one of the things that causes me to avoid the language in my own personal projects.

All of the following are real tools for Python package/environment management: virtualenv; pipenv; venv; pyenv; poetry; pyenv-virtualenv; and pip-tools. There are probably more. Which one is "best" varies by who you ask and with time. If you do a bunch of Python work, you'll probably need most of them installed since different projects use different tools.

The messiness of the whole situation just takes the fun out of it for me. I don't hate Python, but when I code in my free time, I'd simply prefer to use a language that doesn't have those problems.

With libraries that alter your path at runtime. I've encountered comments on issues that implied that this wasn't a complete hack. I have had the impression that this is all just overhead in getting an environment up and running, and not taken for granted that you should be able to achieve repeatable isolation relatively simply.

So I think even if one of those packaging tools worked really well, a library somewhere could undo your best efforts at isolation.

I also don't hate python itself, but I wish the one way of doing things ethos applied to managing dependencies and python versions.

I don't understand what the problem is: 1) you install ONE version of python, more can't coexist in your environment usually 2) then you install your packages in a venv (because python provides tools to configure this)

Only part 2 is a problem of the Python ecosystem, part 1 is a problem of any environment to run binary programs. It applies to C-compilers, node, ..., perl...

Naturally, as most people involved with Python seemed to agree with that view, there was a lot of room for "improvement". And people got their improvement aplenty. Other people just realized that this is a general problem and nowadays instead of keeping around their /opt they now use something like spack. Which allows you to install a range of python-versions just fine. As well as C-Compilers. As well as perl-versions.

I think the unfortunate thing is the lack of unified messaging around this. There are three/four problems that seem interlinked but are not really:

- Managing python versions (no packaging/enviroments) Pyenv is the solution if you don't want to use a global OS version.

- Environments, you don't need anything that isn't bundled with python, `python -m venv env` is the "official solution" and works really well.

- Packaging your app for running in production, `pip freeze > requirements.txt` and `pip install -r requirements.txt` is the "official solution", it works well enough.

- Packaging a lib for public distribution, setuptools... I don't have enough experience there but not many people need to actually use it. But again it is the "official solution"

Pyenv is the only none standard tool that I would recommend.

As far as I'm concerned all of these are redundant or not needed: virtualenv, pipenv, poetry pyenv-virtualenv, and pip-tools.

I wouldn't say those tools are "redundant or not needed". Tools like pipenv add features on top of pip and venv. Stuff like separating development dependencies (e.g. mypy), since you don't need that stuff at runtime. Pipenv will also move transitive dependencies into a lockfile instead of cluttering your requirements.txt (or rather Pipfile in the pipenv world).

> separating development dependencies

I just have:

  requirements-dev.txt (dev only requirements)
  requirements.txt (top level requirements)
  requirements-freeze.txt. (all requirements frozen)

How do you make sure that transitive dev dependencies end up in requirements-dev.txt? Do you manually say "dev dep X depends on transitive deps A and B, so I'll put X, A, and B in requirements-dev.txt"?

I only put top level dependancies in requirements-dev.txt, however you are quite right that a naive `pip freeze > requirements-freeze.txt` would result in it freezing dev dependancies.

It's a problem that I hope PIP will solve one day, but I like to keep tooling simple.

However because it's so easy to create venvs I regally just rebuild from scratch, do a `pip freeze > requirements-freeze.txt` then install dev dependencies (so not freezing dev dependencies). In many ways I think it's better to not mutate your venv and just rebuild if making significant changes.

Fortunately the packaging culture in Python isn't quite the same as with Node and you don't tend to end up with hundreds or thousands of dependancies.

Yea, the JS community's "moar dependencies!" culture is a big downside. But I do think that npm is great and pip should take a lot of cues from it

> It's a problem that I hope PIP will solve one day, but I like to keep tooling simple.

The way I've been doing is to not refer to dev dependencies in requirements.txt and then install the dependencies from requirements.txt before doing a requirements-freeze.txt. I made pip-chill to make the process a bit easier.

pip-compile facilitates this kind of setup and will generate the frozen/lock version for you.

> Packaging your app for running in production, `pip freeze > requirements.txt` and `pip install -r requirements.txt` is the "official solution", it works well enough.

It doesn't though. The default format for requirements doesn't specify which requirements are top-level and which are not. Updating dependencies on a project that doesn't have this distinction is complete pain. It also doesn't work when "production" might mean different environments with different Python versions.

> Packaging a lib for public distribution, setuptools... I don't have enough experience there but not many people need to actually use it.

Actually, using setuptools and setup.py (or a modern equivalent) is very helpful for packaging scripts and commands, defining top-level dependencies or other things. It's much easier to distribute a wheel file to production than a full repository.

Fair enough, I actually do `requirements.txt` and `requirements-freeze.txt` for top level and then everything. But honestly it works very well.

Most of the tools you mentioned appeared precisely because the basics with Python don't work that well. I had a project that started similarly, and in the end I had to use to pip-tools to actually separate top-level requirements and lockfiles for different "flavors" of the package (production and dev dependencies, for example). For new projects I would seriously consider poetry because it manages those two concepts directly. There's also the problem of changing requirement files and the environment not being updated to match 1-to-1 the files, either because someone installs something to test and doesn't put it in the requirements file or because you get an updated file from a coworker and forget to run the update. Even then, pip will not delete packages that it doesn't find on the files.

It's a mess once you start digging a bit, Python dependency management is extremely basic. It's no wonder why alternative tools appear, they're not redundant, they actually fix pain points in the default tools.

Completely with you, in complex projects the tooling can be improved and those tools attempt to do that. However going back to what the OP was suggesting about how the situation puts them off using Python, I do think it's the messaging around this that's the problem. Python, the community, need to unify on a message around official tools. There will always be edge cases, but the noise generated by them can give the impression the ecosystem is broken when it really isn't.

> It doesn't though. The default format for requirements doesn't specify which requirements are top-level and which are no

Shameless plug: you can `pip install pip-chill && pip-chill > requirements.txt` and work as usual. If you feel brave, you can `pip-chill --no-version > requirements.txt` and check if your thing works with the latest of everything.

> It doesn't though. The default format for requirements doesn't specify which requirements are top-level and which are not. Updating dependencies on a project that doesn't have this distinction is complete pain. It also doesn't work when "production" might mean different environments with different Python versions.

This is why (imo) you should include immediate dependencies in setup.py install_requires, and use requirements.txt for the "lock file" that contains all dependencies.

Wow - someone shares my view!

I use pyenv + direnv to automatically configure my Python interpreter and virtual environment for each repo. It is pretty painless.

In addition to the redundant tools you already mentioned: conda.

Conda isn't redundant if you have multiple C-level dependencies like pretty much every DS project in existence. I wish it were, but it's not.

I have to agree with the OP here, Python's dependency management story is just awful.

> - Environments, you don't need anything that isn't bundled with python, `python -m venv env` is the "official solution" and works really well.

Except when you run on Debian based systems, where you need to install the module separately for the OS Python

    $ python -m venv env
    /usr/bin/python: No module named venv

Solution: don't use Debian.

IMO the way most Linux Distro package managers install stuff is absolutely awful (system wide, no isolation, shared libs, shared PATH...)

That's one package you need to install (and that you only install if you'll be developing or testing Python packages). You'll want to install a lot more stuff (python-dev, build-essential) in that case anyway.

OTOH, if you will just use the OS, you won't need venv, as Debian packagers make sure everything works out of the box without it.

One problem with pip freeze (and pip-tools) is that it won't capture platform-specific dependencies. For example, ipython depends on appnope only on macos. If you build your lock file on Linux or Windows, the appnope version won't be locked.

Poetry solves this to an extent by including these dependencies along with their markers in the lock file.

I don't disagree with you but pyenv is just nvm for Python. You use it to manage multiple Python versions without touching the system Python, which tends to get people into trouble.

You don't need any of those tools. One can use virtualenvs to avoid global Python pollution, but that's it.

I've used poetry, pipenv and others... Nowadays my opinion is that I want my package managers to be simple.

For each app, I mantain requirements.txt, requirements.dev.txt, requirements.test.txt... I edit those files manually and only use pip freeze for pinning requirements.freeze.txt

For libs, flit + pyproject.toml is a breeze.

None of the the tools are needed. You could compile Python 3.10 for each project on separately and just set your PATH to use the one you want. But pyenv makes life a lot easier.

Come on..it's very, very far from a debacle. It has strengths and weaknesses, sure. But a debacle?

pip, pip-tools (to generate a lock file), and virtualenv are all I've needed for years now.

I just started using pyenv to install Python on Ubuntu. Before that I was using the deadsnakes ppa.

The tools are there to use as needed. You don't have to install or learn them until...you have to. Which may very well be never.

If you avoid Python and all its benefits because of its packaging situation, it's really your loss.

I just hope no one reads this and gets scared off thinking that this top voted comment matches reality. It's not that hard to get your bearings and the Python foundation has invested heavily into the packaging ecosystem the last few years.

For example: https://packaging.python.org/en/latest/

Unless you have multiple C-level dependencies and/or need to install a Google created framework :(

I just use pip and, sometimes, virtualenvwrapper. You just pick one that works the way you want and be happy.

Having said that, I dislike the way pyenv works by adding proxy interpreters that are globally visible.

Is this any different/worse than JS/node ecosystem? Npm vs yarn vs bower. Webpack vs rollup, etc.

It’s a total mess. But build and runtime environment are a weak point for many, many languages.

Very much agreed. I briefly tried to use poetry, tox, and pytest, and even using the support packages that are supposed to make poetry and tox play nicely together, it was a massive failure. tox would claim to be running tests with python3.8, say, but actually it was still using the default system python3.10, etc.

I now refuse to use any packaging tools for anything other than generating a requirements.txt, and refuse to use any version matrix tools like tox. I don't have a preference for/against pyenv for sourcing python interpreters (I'll use an AUR or scoop sometimes), or poetry or pipenv or anything else for specifying dependencies, but whatever method is used, I will always only use them to export a requirements.txt, and I will manually create my own venv for each interpreter, and run anything I want inside each venv via oneliners like:

    bash -c '. "$0/bin/activate" && exec "$@"' venv38 pip install --upgrade pip setuptools wheel
    bash -c '. "$0/bin/activate" && exec "$@"' venv38 pip install -r requirements.txt
    bash -c '. "$0/bin/activate" && exec "$@"' venv38 pytest --doctest-modules -v

Usually I'll wrap that `bash -c` preamble into a simple with_venv.sh script, and make a similar with_venv.ps1 for Windows. Create a Makefile with targets for those commands for each interpreter you want to test, and you can run everything in knowledge that you'll always know what's happening, instead of relying on stacked magic.

You're missing Pyflow which is writtenin Rust. But alpha at least. But wait, there's more…

One thing i like about pyenv that isn't very well documented is the fact that you can easily install the latest release from a specific version by using the :latest suffix. For example:

pyenv install 3.8:latest

Will install the latest 3.8 Python version.

wow you're right, that should be documented, I'm going to use this every time now. I've always been annoyed at having to learn what the latest patch versions are.

I think you collectively saved hundreds of pyenv users from some minor suffering! Thanks for the tip!

And yet I wonder why this is supposed to be an integral part of the Python ecosystem.

Q: Now you want to build your Fortran extension with a compiler supporting OpenACC. What now?

This should show relatively clear that this tool is just an edgecase of a quite general problem. I recommend a general solution (like spack).

Because a lot of projects, probably the great majority of projects, do not need this level of control.

If you need that level of control, use Spack or Nix or whatever.

Whenever threads like this come up, people have a weird tendency to hold Python on some kind of pedestal above other languages.

I can't think of a single language package manager or installer or version manager that makes even a faint attempt to control system level dependencies like a Fortran compiler. Why is Python somehow bad for failing to manage this? The problem is and should remain out of scope for any particular language tool chain.

Yes, I agree on that, spack can lock everything except the kernel and libc.

I also don't think this is a failure of "python" as some others have raised that in the vincinity. Managing binaries is just out-of-scope and should be left to another tool - such as given with pyenv. It's solving this problem in a narrow way, which most commenters find confusing and I find a bit useless compared to alternatives!

The reason that Python gets held to this standard is because data science/analytics etc has lots of C and Fortran dependencies and this interacts terribly with the YOLO attitude of python dependency management tools (pip did not check for dependency compatibility until 2020, and it still doesn't keep your environment consistent.

PSA: You don't need to use Pyenv (or asdf, or anything else...) on Linux. It just creates a mess.

If you need to install multiple versions of Python that are not available from your distribution packages, simply download the code, install it to a place like /opt and then point your virtualenv to that version when you need it.

Here's what's needed:

> ./configure --prefix=/opt/python-<VERSION>; make; make install

This is not more complicated than what pyenv asks you to do (read the article).

Once you have all the versions installed, no need to use a tool to switch between them. When you use virtualenv (which you should, and no need for pipenv, poetry and the rest), just do this to create a new virtualenv:

> /opt/python-<VERSION>/bin/python -m venv /path/to/where/you/want/to/store/your/virtualenv

This will pick up the correct version of Python. Once you activate the virtual environment, that's it.

There's no need to use anything besides what default Python installation already gives you.

PS. What about the dependencies? Glad you asked: `apt build-dep python3` on Debian/Ubuntu or `dnf builddep python` on Fedora. No need to go through them manually or copy/paste some list off someone's blog and hope it's still valid.

PPS. All this applies to Python 2 with minor tweaks, but hopefully you're not starting new Python2 projects in 2022.

There’s a command in pyenv called “python-build” that more or less automates the Python compilation part. Most don’t realise it (it’s hidden from the general UI) but that’s where most pyenv functionalities come from.

I agree the rest of the pyenv (shims, shell magic etc.) tends to make a mess.

This, a million times. Extra tools means extra magic and magic is always bad.

You can use Pyenv entirely as a build/version manager if you want. Just don't use `pyenv activate` and point your tools to the absolute path of the Python interpreter that you want to use.

In recent versions, the shims are installed to a separate directory, which you can omit from your PATH if you never want to use them.

> You can use Pyenv entirely as a build/version manager if you want

Usually the distros suffice for that. There are very few cases where it's worth to roll out your own Python.

> In recent versions, the shims are installed to a separate directory

This is a huge progress!

This should be the top comment. I might add direnv to automatically activate the virtual environment when you CD into the project directory, and to automatically deactivate it when you CD out of that directory

Unfortunately, you might have to use tools like a CI/CD system or a runtime environment (AWS Lambdas, for example), where none of this flows down correctly.

Edit: As a concrete example, say you need a C based python package to work locally, but also build within Gitlab and be deployable to an AWS Lambda. There's a gap there where Python really wants absolute paths that don't change.

I highly recommend Pyenv, it's the best way to manage Python versions.

My workflow on a Mac is to use Brew to install Pyenv and then Pyenv to install Python versions. Similarly I use Brew to install NVM and then NVM to install and manage Node versions.

For python virtual environments I go with vanilla `python -m venv env`, it works well and I like that the `env` directory is under the project I'm working on.

> My workflow on a Mac is to use Brew to install Pyenv and then Pyenv to install Python versions. Similarly I use Brew to install NVM and then NVM to install and manage Node versions.

Might I suggest asdf (asdf-vm.com). I use it for managing ruby, node, go and python versions and it has been rock-solid. You can also install it from homebrew.

I switched from installing Python with Homebrew to using asdf and I've been very pleased with the results. With Homebrew, every significant macOS update would break all my virtual environments. With asdf this is no longer an issue. The ability to set local versions of Python for different projects is very nice as well.

I always have different python versions on system level (`$ sudo apt install python3.7 python3.8 python3.9 python3.10`) and use virtualenv to manage versions for each project (`$ virtualenv env -p python3.9`), and that has served me well for years. Could someone tell me why pyenv is superior?

pyenv can compile Python versions that might not be available with "apt install" yet. And you can have multiple minor versions available (3.7.8 and 3.7.9, for instance), if you ever need it.

I don't know if it's "superior", but it helps you manage / install / uninstall / set system Python versions, as well as virtualenv features. That's it.

Thanks. IMO changing system python versions might confuse some people.

I often have to debug not-so-tech-savvy colleagues' python scripts/environments.

If we used pyenv, I could picture me telling them that they forgot to switch system python environment.

If you forget to activate the venv you cannot accidently install packages in your system python.

Asdf provides a very similar workflow, but for many other packages too.

I'm really drawn to Nix for it's power and purity, but until it has a first class way to install specific older software versions the 'worse is better' asdf way wins.

I use miniconda and have had no major issues. I’m curious if anyone here switched from conda to an alternative and had a better experience.

I switched from conda to pyenv after they changed their license

Updates work about half the time for me on mini-conda, but they break the other half of the time. When I look up the issue, I am not alone. I end up re-installing it a lot.

I have had this happen a few times too. I have found the best way to minimize this is to install nothing in the base environment except for conda and make sure you use a different environment for everything else.

I appreciate the time, but I have two responses:

The first, that has always been my practice.

Secondly, the vast majority of solutions offered by the many people who face similar issues don't work for me or, apparently, for other people reading the solution as well.

I use conda for my data analysis environments and poetry+(pyenv) for developing python libraries and application that are intended for running on machines others than (just) my own.

I switched from conda to poetry because everyone was praising it last year. Then I switched back because it's a pain to switch between OSes using poetry. I only use conda + pip install now, works well enough that I don't have to struggle every time I deploy in a new environment.

Pyenv combined with Poetry is still the cleanest way to run multiples envs. Any other suggestions?

miniconda is nice for that too, especially if you want non-python things like cuda.

I had a lot of fun with https://pdm.fming.dev/

Nice, never heard but almost seems like a 1:1 competitor?

I've always found pyenv a bit complex to deal with. I've also encountered some bugs with a fresh debian install and pyenv. To put this in perspective, i'm not the sharpest knife in the drawer (not a programmer, not an adminsys).

I prefered alt install for a specific version (python3.3, python 3.X, etc. with associated pip) and virtualenv. Then doing `python3.X -m venv` i got what i need. VScode or pycharm can bind to it.

Recently i've tryed Fedora and they ship python versions compiled so you just need to use the package manager. It's easier and it deals with updates. I stick with that now (i'm migrating from deb to fedora for my servers).

I like and use poetry, it generally just works.

It is however slow as molasses and its error messages are sometimes worse than a stack trace.

I use pyenv + direnv. If my project uses 3.10.3 I just add `layout pyenv 3.10.3` to my .envrc file. This creates a venv in the `.direnv` subdirectory. When I `cd` into the dir my environment is automatically configured to use the virtual environment and when I `cd` out the configuration is removed.

If you want to use poetry + direnv, that's also an option.

https://github.com/direnv/direnv/wiki/Python#pyenv https://github.com/direnv/direnv/wiki/Python#poetry

The maintainers of the Python packaging have just officially adopted Hatch:

https://hatch.pypa.io/

Hatch aims to do what Poetry does, but is strictly compliant with Python standards.

> The maintainers of the Python packaging

Who?

> have just officially adopted Hatch:

Do you have a link?

The Python Packaging Authority, the team that maintain setuptools etc. for core Python.

There's a discussion here:

https://discuss.python.org/t/hatch-1-0-0-is-available/15359

Oh great thanks for the link :)

What’s wrong with running `python3 -m venv venv` and then activating/deactivating the venv as you go? Feels like if there’s an out-of-the-box way for keeping separate environments with scoped dependencies, then we shouldn’t fragment things further with yet another approach?

The wrinkle comes in when you need different versions of Python for different projects. e.g. Python 3.6 for one, 3.10 for another.

So what’s the actual problem? You can install multiple python versions and use the correct one when creating the venv.

pyenv helps you install and manage those multiple python versions.

I don't use pyenv and have multiple python versions installed (and use them via venv per project). If you don't need to designate one as a global/system Python (which I don't), is there any reason to use pyenv?

If your process to install a specific Python version for a project, independent of system Python, is as fast and simple as `pyenv install 3.9.7`; and if you don't work in a team where having `.python-version` file in the project will download, install and set the exact Python version required: you don't need pyenv.

It's solving a completely different problem, Pyenv is entirely about managing versions of Python itself, nothing to do with venvs or packages.

Say you need to have a venv but for a specific Python version:

  pyenv install 3.8.5
  pyenv shell 3.8.5
  python -m venv env
  source env/bin/activate

Now your venv is for 3.8.5, and running `source env/bin/activate` will always activate that version within the environment.

why did they call it pyenv when it has nothing to do with python environments, well at least the existing python virtual environment nomenclature

`python3 -m venv venv` will use whichever minor version of Python 3 that `python3` currently points to. If you are working on multiple projects that span different Python 3 minor versions, pyenv makes the process of switching between versions much easier. You would activate the specific version you need with pyenv then use `python -m venv venv` to create the virtual environment using that specific version. It's not one or the other, but they work nicely together.

Sometimes you will need a different version of Python for project A versus project B.

A common scenario: Python 3.10 comes out and you want to make sure your project (and all of the upstream dependencies, libraries) work with it. With pyenv you get virtualenv isolation as well as isolation of the language itself. You can have a unique Python for each project, not just a unique virtualenv sharing the same install. This also makes benchmarking kinda fun, you can jump back and forth between versions and run the test suite to gauge if any big changes have occurred time wise.

Personally, I love idempotent environments that I can create and destroy on a whim. Something go wrong? Package install failed half way through and left clutter? Destroy the directory, make new pyenv/virtenv and install again. Leaves little room for doubt.

I used to share global pythons but as I started to take on more and more consulting projects it became imperative to keep them completely isolated.

Ok, got it. Haven’t felt the need for it because I’m pretty conservative regarding language features.

But I definitely see why you‘d need it. It’s just a bit sad to me that there’s yet another tool with a potentially confusing name, orthogonal to the system Python. “Wanna get started with Python? Sure just install another Python (ignore the one that comes with your OS!) using pyenv, then it’s just a matter of getting a virtualenv and pip-installing your requirements.txt! Or was it Pipenv? Or setup.py even?“. Oh and it’s almost guaranteed that your IDE will pick the wrong instance.

It‘s just too much. I see non-Python enthusiasts struggle with this stuff on an almost dsily basis. Especially the ones who only have exposure on an irregular basis.

I wish OS distributions would just STOP providing Python packages at all.

So much about Python the language and its standard library is great. But many peripheral things (the Python 2->3 debacle first and foremost) are just badly executed.

Last I checked, pyenv was dependent on absolute paths that can't change. This makes it very difficult to use with things like CI/CD pipelines that copy repos into temporary directories. There used to be a "--relocatable" switch, but it's gone now.

https://github.com/pypa/virtualenv/issues/1549

Edit: Ah, yep, somehow read this as virtualenv.

pyenv != virtualenv though?

My frustration with managing different Python versions is what motivated me to finally go out and learn Docker, after running into scenarios where I needed to manage different versions of Python with different versions of each library.

pyenv will do this with the virtualenv integration, so you get the best of both worlds:

    pyenv virtualenv <python-version> <env-name>
    pyenv activate <env-name>
    pip install ...

Not really an option on a Mac. Docker is so incredibly slow not only because of the virtualisation layer, but because of file system performance (Python hits a lot of files on startup).

It’s also not that complex at the end of the day. You have two things involved in managing python versions: the executable you’re running, and the PYTHONPATH.

I actually don’t understand why people find that so complex. A virtual environment is: an alias to a python executable; a folder called ‘site-packages’ which is then added to the PYTHONPATH; and a script to set you PATH and PYTHONPATH.

I get what you mean but being on a Mac I now avoid Docker for development, it seemed to cut my battery life in half whenever it was running.

Is this something distinct from pipenv? Gee, python packaging is a mess..

Pyenv is a bit lower-level and works well with requirements.txt. Pipenv can use pyenv to install Python venvs and brings its own Pipenv and lock files.

I believe pipenv is a wrapper around pip and pyenv.

Pyenv is brilliant, defiantly recumbent it as the standard way to manage python versions. It's basically nvm but for python.

Pyenv has nothing to do with pipenv.

Pyenv is mainly for installing versions of Python. It's useful if you work on multiple packages that each needs a different version of Python, or if you have one package that you test with multiple versions of Python. (Alternative to apt/yum/brew or manually downloading Python from python.org)

pipenv is for installing Python packages for your project. (Alternative to pip, poetry, etc)

Never tried Pyenv, but I can highly recommend Mamba a fully Conda compatible package manager that is insanely fast solving dependencies.

Installing python: `mamba install python=3.9`

How does mamba do inside Docker containers? Does it have the same issues that the conda activate command has?

My personal approach has been to use conda for binary management and a place to stuff libraries (could just as easily use venv here), but the key tool I'm using to pin libraries and avoid transitive dependency drift is pip-compile.

Frankly the invocations are so arcane for these various supporting tools that I have been distributing a standard Makefile to projects to hide the complexity - "What's old is new again"!

I like pyenv and install it on various servers, sometimes for the latest version of python 3.xx, sometimes for 2.7, without depending on the version the OS provides.

To run it from cronjob and other places where environment is harder to define, the following can be used:

0 2 * * * /home/user/.pyenv/shims/python /home/user/bin/script.py

Unpopular opinion: tools other than virtualenv are for people who do not understand how virtualenv works. On the other hand: people who do understand how virtualenv works, know it very well we: don’t need the other tools than virtualenv.

Do you mean venv which is part of standard library or virtualenv (separate library)?

AFAIK venv is a subset of virtualenv brought into the standard lib around python 3.3.

I’m talking about the “full” virtualenv. This one: https://pypi.org/project/virtualenv/

virtualenv doesn't let you manage installations of different python versions. This is the problem that pyenv solves.

And this is not a Python problem either. It's a general problem.

IMHO it’s solved by installing different python versions using the OS package manager, like `dnf install python3.8 python3.9`, and then creating virtualenv using whichever version you need, like `virtualenv -p python3.9 venv`

You'll be limited to whatever python version is packaged on the host operating system. pyenv can let you test against development versions of python if needed, or released of python that are either new enough (or old enough in some cases) that they aren't available on your operating system's package repositories.

You can `make dist` on the python repo to build the rpm of whichever branch/commit you want to use. Pre-built ones are also available from nightly build. I still don’t know why you need a separate tool for that.

What is with the endless stream of basic Python articles? Perhaps the Python ecosystem is so complex that it requires constant education.

It's like CSS — You unlearn it quickly if you don't train the Python muscle.

It's a mess... there is no convergence on package managers or anything and everything sucks. Node/nvm at least was forced to fix its s by how bad it originally was.

In Python everything sort of almost works good enough for each use case, so half thought solutions keep staying in each niche...

Reply on news.ycombinator.com