Every time I get the urge to pick up Python, I find the packaging situation so off putting that I quickly find some other rabbit hole to dive into. I find emacs and vim add on configuration easy and transparent by comparison.
Maybe you are an accidental accelerationist? Using the system Python is fastest way to a broken system and a broken Python.
I always use the system Python plus virtualenv and that works fine, it's certainly never broken my system...
I don't even know by what mechanism that breakage would happen.
If you just run "pip install" doesn't literally tell you to use the --user flag or virtualenv?
I guess it's probably an easy mistake to make.
Debian is particularly bad because they'd comingle python 2 and 3 packages in the same filesystem locations if you used pip out of the box.
That was a horrid discovery on my part. I assumed, coming from java where this sort of thing has been a long solved problem, that no one would be daft enough to do something like that.
Most projects state something like "run pip install whatever", which people will then do. If you're lucky, it will ask for sudo. If you're even luckier, you stop to think before entering your password.
That default Python env means I don't ever modify the system python for any reason. I type `python` and I get my own Python3 from the base.env
If you use homebrew, there are two settings that help keep your old python binaries around so that venvs that reference those old envs don't break.
HOMEBREW_NO_CLEANUP_FORMULAE, this takes a lists of packages to not cleanup
HOMEBREW_NO_INSTALL_CLEANUP, prevents old packages from getting removed
I actually far prefer the Python method to, say, Go's GOPATH. But, I far prefer Rust/Cargo to either.
I wish Python got a standard integrated solution for package management that works.
pip doesn't cut it in my book since it doesn't let you specify dependencies reproducibly. It either doesn't support lockfiles or ONLY supports having a lockfile, without dependencies.
venv also doesn't cut it since you have to remember to explicitly activate it and keep track of which venv is activated right now.
If we take a look at the much-maligned Node.js+npm, it's still far better than what built-in tools in Python let you have. Yes, Node.js doesn't provide full isolation from globally-installed node_modules, but at least it supports a local node_modules directory and lets it take preference. Notably Node.js+npm, with all its warts, doesn't suffer from the the two issues I've mentioned above.
I think this should be emphasized more, I don't think it's just a matter of preference, not separating packages between projects using virtualenvs will land you in a world of hurt as soon as you want to update or uninstall any of them and they're hopelessly entangled with system packages and other projects.
What is impressive, is how so many in the industry have forgotten about them.
Save in requirements.txt. Includes sub dependencies that are system specific.
Install from requirements.txt on different machine and get errors.
Uninstall dependency and save to requirements.txt.
Look at requirements.txt and see that sub decencies are still there.
What is the right way to avoid these issues on Python?
Keep your high-level dependencies in requirements.in.
Create a virtual environment using venv.
Run `pip-compile resolver=backtracking`. It autogenerates a requirements.txt file with all dependencies and sub-dependencies and their versions. This essentially acts as a lock file from other languages/frameworks.
Install from the autogenerated requirements.txt file in the venv virtual environment.
If a dependency changes, change in requirements.in, recompile the .txt file, and reinstall.
This is new for me. I used `pip freeze > requirements.txt`
* use a virtualenv, so that dependencies are installed per project and not globally
* only specify my top-level dependencies in a requirements.in file, and let pip-compile (a dependency, part of pip-tools) compile my requirements.txt
So far it has served me well and I've not encountered any error due to this method.
I cannot claim it is "the right way" though, just something that works for me.
Or junk all this and just use poetry, which manages both the abstract dependencies (pyproject.toml) and concrete ones (poetry.lock).
However: poetry still falls short in managing the python runtime, I am continuously having to divert time to help our data scientists untangle the messes poetry makes with virtualenvs and their local python setups.
Also, they broke backwards compat on the lock file format? So now devs running newer poetry versions break projects for devs on older versions because the lock files aren’t compatible?!