Hacker News Reader

Comment by victornomad | Hacker News Reader

Any recommendation on how to setup an environment with Jupyter and nvidia/cuda? Is it worth to use a docker container? or installed system wide?

View on HN

I once tried running jupyter inside docker...the downside i saw was conda install of jupyter is around 1.2 gb, making the container pretty heavy...if you are fine with this then all thats left is to expose the port to the container on which you want your jupyter server to run and changing some config after generating jupyter_config file for remote access.

Hi, I’ve been working on setting up JupyterHub on a DGX-1 for the past week. It’s straightforward, you run JupyterHub on your machine and you have it launch servers for each user with nvidia-docker.

At work we use conda which works great, though nvidia libraries are usually installed by admins system wide. But for anything python related, conda is perfect.

Thanks for the answer :) what is the advantages of using conda vs installing the packages by hand? Tried to document my self a bit before but I still dont know the real advantages in a modern Linux system

Conda is "just" a distribution and package management tool. As a beginner it is a good idea to just go with it, but installing all packages using pip is just as good, in the end. For reference: https://www.xkcd.com/1987/

In bioinformatics there is a trend for systems like qiime2 to be basically impossible to install via pip and to have conda and docker be the only options. In part this is because rarely do bioinformatics pipelines rely only on python but on a multitude of existing programs that need to be installed.

There are at least three:

1. Anaconda sells support that many companies will value, https://www.anaconda.com/support/

2. Anaconda checks the installed versions of packages in the distribution are compatible.

3. Conda has an “environments” feature so a developer/scientist can switch between many, project-specific development environments, https://docs.anaconda.com/ae-notebooks/user-guide/adv-tasks/...

Edit: Also, there is a distribution for Windows that’s handy when your employer has you using Windows. And, depending on your software approval process, it can be convenient to get one package approved (Anaconda) instead of every package Anaconda includes.

Conda gives you two real advantages: the first is straightforward virtual environments (you could also use venv) and secondly better dependency management than pip in some cases.

There are certain packages which aren't available on pip that conda-forge provides e.g. for a while OpenCV wasn't available through pypi.

Remember you can use pip within conda, and you can install directly (e.g. setup.py) within a conda env.

Docker is mostly useful if you want to mix and match different versions of CUDA/cudnn etc. If you just want to run an isolated Python environment, then Conda will do the job.

Reply on news.ycombinator.com