Creating Python Virtual Environments with Conda: Why and How | Hacker News Reader

Creating Python Virtual Environments with Conda: Why and How

heartbeat.fritz.ai

51 pointsAnoetar21 comments

This tells me why a virtual environment is good, but not why a Conda venv is good. Am I missing something?

Why not just use pip?

Conda can manage a few things beyond just Pip. Things like binaries, system libs, R installs.

I'm not entirely convinced that's a good thing, but it's an easy path to managing those sorts of libraries alongside your Python libs.

One reason it can be very useful is that a conda environment gives data scientists a super easy way to Dockerise their code.

Binder (https://mybinder.org/) is a good example of how well this can work - anyone can reproduce your work using e.g. Jupyter hosted on a binder, build from a conda env.

Conda takes care of a lot of other useful things. For example, it will install cuda and cudnn for you. It will also install mkl. So much easier than pip + many other steps.

Conda virtual environments and Pip are orthogonal. Pip can be used to install into venvs. But as to why use conda rather than just the standard library venv module (python3 -m venv venv): conda can easily manage multiple python versions. In my experience, it's the most hassle-free option compared to something like pyenv. I personally use conda just for managing venvs, and don't use its package manager functionality at all.

FWIW our use case is the same:

- Conda to manage multiple python versions on 1 box:

  $ conda create -n env27 python=2.7
  $ conda create -n env37 python=3.7

- Pip to manage packages in the environment:

  $ source activate env27
  $ pip install -r requirements_27.txt

But why would anybody need a Python version other than the latest? Aren't they backward-compatible within a major version? Won't everything written for Python 3.7 work on 3.8 and everything written for 2.6 work on 2.7?

I'm by no means a Python expert but I recently had a small project where I needed to convert the output of an SQL query to a parquet file, and I settled on using fastparquet to do so.

I first tried using pip under a venv to install it, and I ran into dependency hell. I needed various dev libs and build-essential on my ubuntu box and I ultimately couldn't get it to compile llvmlite correctly (linker issuers and way too many LLVM versions).

I finally gave up and tried miniconda on a fresh box per some advice here on HN. I simply installed it, ran "conda create -n myenv fastparquet" and it just worked first try. No extra effort involved.

I mean, if you've ever hosed your ubuntu (etc) installation by replacing your system's python2 with python3, you already get the why.

Although the article doesn’t really go into it, the best thing about using conda is that it obviates the need for pipenv or poetry.

Is there a global or user config that overrides the environment config file?

It seems less valuable if the prefix is hard coded to the specific user. To a lesser extent even specifying the name may be too limiting.

Do you mean the condarc file? You can do `export CONDARC=./project_condarc`

No I mean in the config file example the PREFIX is set to a static path including a user directory. This means if anyone checks out the config file and tries to run it the path will not exist. Is there an override somewhere that provides the desired prefix at a user level? Something like ~/.condarc or /etc/condarc?

Ideally I would set the prefix for Conda for myself only once (~/.condarc?) and the source controlled config file would then extend that config.

I would expect the conda config file in a project to say nothing at all about the prefix unless it is relative to an existing prefix I set elsewhere.

This way I don't have to manipulate my environment and edit config files every time I run conda.

I should probably just RTFM.

As already stated conda and pip both enable you to create separate environments. A major advantage of using conda over pip is that conda has additional channels, such as bioconda and conda-forge, that enable software besides python or R to be installed. Through bioconda a user can install bioinformatics unix software with a single command. See https://conda-forge.org/ and https://bioconda.github.io/.

Also, you can use pip to install software if you have the anaconda distribution installed. As far as I know it does not work the other way around.

bring back stow!

It never left.

Or, if you actually know how to use the Unix command line in non-trivial ways, you can just use pip.

Not sure, why this article is on HN front page.

Because people are upvoting it.

Thanks for the snark, but my point was is this article front-page worthy?

Define front-page worthy.

Anything that is on the front-page is by definition front-page worthy otherwise it wouldn't have made its way there.

Reply on news.ycombinator.com