Hacker News Reader

Comment by bxparks | Hacker News Reader

What is the best way for an author of a Python package to develop and test on multiple versions of Python (e.g. 3.6, 3.7, 3.8, 3.9, 3.10), and be able to switch easily among the various Python versions? Every time I try to research this, I get lost in the chaos of Python packaging and environments.

Currently I do a `pip3 install -e .`, which uses the default Python provided by the OS. Then I hope that my continuous integration matrix catches any problems in the other Python versions.

View on HN

You need `pyenv` and `nox`. Just read this entire guide https://cjolowicz.github.io/posts/hypermodern-python-01-setu...

I think tox was designed to solve that problem.

https://pypi.org/project/tox/

This, absolutely. If you're running tests for building a package, here's an entire "tox.ini" file for running all your tests on multiple python versions:

  [tox]
  envlist = py27,py36,py38

  [testenv]
  deps =
     pytest                 # Install extra packages for testing separate from what's in setup.py or whatever
  commands =
     pytest                 # Whatever test command your project uses

That's really it for bare minimum. If you have pytest configuration, they can also go into tox.ini under a [pytest] section to keep them in the same place.

Instead of installing tox globally you can even install it to a separate virtualenv, activate it, then run "tox". It figures out the right thing to do.

If you're not building a package:

  [tox]
  envlist = py27,py36,py38
  skipsdist = True          # Don't attempt to build a package / pull from setup.py

  [testenv]
  deps =
     -r requirements.txt    # Pull in project requirements
     pytest
  commands =
     pytest

If you scroll down in the linked article it says:

> ALWAYS use an environment! Don't install into your global interpreter!

...

> When you need to create an environment for a project I personally always reach for venv and virtual environments. It's included in Python's stdlib so it's always available via python -m venv (as long as you are not on Debian/Ubuntu, otherwise you may have to install the python3-venv apt package)

I think this is good advice, and it's pretty easy to do, and you can have named environments (via directory names) for each version you can switch between easily.

Very occasionally some python libraries will use a globally installed system library or a particular version which changes per python version (I think maybe OpenCV has this problem on Windows or something?) and so then you have to reach for containers.

Environments don't help you if you want to use different versions of Python.

Just want to thank everyone who responded with suggestions. I think it's amusing that there seems to be at least 5 different solutions to this problem: pyenv, nox, tox, asdf, docker.

A lot of people will recommend pyenv, but I find this ends up causing obscure problems as it messes with your PYTHONPATH.

I like to just install alternative versions from source. Download the tarball you want from python.org and run

    ./configure
    make
    make altinstall

and it will be installed separately without overwriting any other version. Then you can just create a venv to use with that version like

    python3.6 -m venv py36

pyenv is my tool of choice here. It allows one to easily install multiple versions of Python from source and switch among them.

It has mostly avoided feature creep, which is a big plus among all the competing solutions.

Not an expert in python, but there is pyenv. I personally use asdf, which is great because it manages versions for multiple language runtimes.

asdf is good, and supports more than Python

https://asdf-vm.com/

use pyenv (https://github.com/pyenv/pyenv) to manage multiple python versions (e.g. install 3.6, 3.7, etc) and then use virtual environments

or use containers and a docker-compose file to run your test suite across versions

Reply on news.ycombinator.com