Comment by westurner | Hacker News Reader

Comment by westurner | Hacker News Reader

There are Alpine [1] and Debian [2] miniconda images (within which you can `conda install python==3.8` and 2.7 and 3.4 in different conda envs)

[1] https://github.com/ContinuumIO/docker-images/blob/master/min...

[2] https://github.com/ContinuumIO/docker-images/blob/master/min...

If you build manylinux wheels with auditwheel [3], they should install without needing compilation for {CentOS, Debian, Ubuntu, and Alpine}; though standard Alpine images have MUSL instead of glibc by default, this [4] may work:

  echo "manylinux1_compatible = True" > $PYTHON_PATH/_manylinux.py

[3] https://github.com/pypa/auditwheel

[4] https://github.com/docker-library/docs/issues/904#issuecomme...

The miniforge docker images aren't yet [5][6] multi-arch, which means it's not as easy to take advantage of all of the ARM64 / aarch64 packages that conda-forge builds now.

[5] https://github.com/conda-forge/docker-images/issues/102#issu...

[6] https://github.com/conda-forge/miniforge/issues/20

There are i686 and x86-64 docker containers for building manylinux wheels that work with many distros: https://github.com/pypa/manylinux/tree/master/docker

A multi-stage Dockerfile build can produce a wheel in the first stage and install that wheel (with `COPY --from=0`) in a later stage; leaving build dependencies out of the production environment for security and performance: https://docs.docker.com/develop/develop-images/multistage-bu...

Interesting! I use miniconda extensively for local development to manage virtual environments for different python versions and love it. I hardly ever actually use the conda packages though.

I assume the main benefit of using these images would be if you are installing from conda repos instead of pip? Otherwise just using the official python images would be as good if not better

Edit: I guess if you needed multiple python versions in a single container this would be a good solution for that as well

Use cases for conda or conda+pip:

- Already-compiled packages (where there may not be binary wheels) instead of requiring reinstallation and subsequent removal of e.g. build-essentials for every install

- Support for R, Julia, NodeJS, Qt, ROS, CUDA, MKL, etc.

- Here's what the Kaggle docker-python Dockerfile installs with conda and with pip: https://github.com/Kaggle/docker-python/blob/master/Dockerfi...

- Build matrix in one container with conda envs

Disadvantages of the official python images as compared with conda+pip:

- Necessary to (re)install build dependencies and a compiler for every build (if there's not a bdist or a wheel for the given architecture) and then uninstall all unnecessary transitive dependencies. This is where a [multi-stage] build of a manylinux wheel may be the best approach.

- No LSM (AppArmor, SELinux, ) for one or more processes in the container (which may have read access to /etc or environment variables and/or --privileged)

- Necessary to build basically everything on non x86[-64] architectures for every container build

Disadvantages of conda / conda+pip:

- Different package repo infrastructure to mirror

- Users complaining that they don't need conda who then proceed to re-download and re-build wheels locally multiple times a day

Additional attributes for comparison:

- The new pip solver (which is slower than the traditional iterative non-solver), conda, and mamba

- repo2docker (and thus BinderHub) can build an up-to-date container from requirements.txt, environment.yml, install.R, postBuild and any of the other dependency specification formats supported by REES: Reproducible Environment Execution Standard; which may be helpful as Docker Hub images will soon be deleted if they're not retrieved at least once every 6 months (possibly with a GitHub Actions cron task)

Quite a few conda packages have patches added by the conda team to help fix problems in packages relying on native code or binaries. Particularly on Windows. If something is available on the primary conda repos it will almost assuredly work with few of any problems cross-platform, whereas pip is hit or miss.

If you’re always on Linux you may never appreciate it but some pip packages are a nightmare to get working properly on Windows.

If you look through the source of the conda repos, you’ll see all kinds of small patches to fix weird and breaking edge cases, particularly in libs with significant C back ends.

Here's the meta.yml for the conda-forge/python-feedstock: https://github.com/conda-forge/python-feedstock/blob/master/...

It includes patches just like distro packages often do.

Reply on news.ycombinator.com