I have just started getting into python as I'm experimenting with pytorch and after a lot of googling I'm still trying to figure out how to use conda properly. Could anyone point me to the best way to manage python environments and packages?
Everyone will tell you of the "best" (i.e. their favorite) way to handle packages and dependencies. Usually mixed with some sandbox framework. Everything will be broken in some way: nonstandard, convoluted, too slow, cannot handle corner cases, or just way too difficult to understand. Even those labeled "for humans". Everyone will agree the situation is a mess "except for this one way that works": never trust that way, it doesn't really work past trivial cases.
Of course this is just my experience. But I feel after years of battling with Python and trying different things, I've earned the right to say this.
It encourages to set up project specific definitions which are saved in the local pyproject.toml file. Keeping everything local and project specific, including the env definition, turns out to be a fantastic boon to reproducibility and sanity of mind.
- `conda env create|list|remove` commands to manage them
- `conda activate <env name>` to enter your environment
- `conda deactivate` to return to your base environment
If you want to use a conda environment in jupyter, you must install ipykernel and create a kernel definition for your environment:
conda activate <env name>
mamba install -c conda-forge ipykernel
ipython kernel install --user --name <env name>
---
1. Create a virtual environment named venv
cd $PROJECT_FOLDER
python -m venv venv
source ./venv/bin/activate3. Define and install dependencies
nano requirements.txt
pip install -r requirements.txthttps://docs.conda.io/projects/conda/en/latest/user-guide/ta...
(Lots of the other advice here—mamba excepted, and that might be a good choice—is good for python in general but potentially will cause problems if you are otherwise using anaconda.)