Hacker News Reader

Herein lies my problem. If I want to start a Node project I run `npm init` and then `npm install --save` to my heart's content. If I somehow manage to mess it up I just delete node_modules/ and install again.

If I want to start a Python project I have to set up venv and remember to put relative paths in front of every command or else it'll run the system version. Sounds simple, but it's still something to always remember.

Just source the activate script and it'll prepend the correct path so that you don't need to do anything else.

You have to use something like anaconda. It's basically the same as npm_modules (sorta).

Use virtualenvwrapper.

$ mkproject foobar

$ workon foobar

You're set

Also: set PIP_REQUIRE_VIRTUALENV=true

You won't be able to install with pip unless you are in a virtualenv.

    1. pip install --user and sudo pip install are fine actually, they will not interfere with venv, they can co-exist just fine.
    2. yes
    3. probably do "source bin/activate" first, then run 'pip install -U pip'
    4. just run pip install whatever, no need the full PATH
    5. just run python directly, no need the full PATH
    6. run 'deactivate' when you're done for now, 'source bin/activate' when you want to continue/resume sometime later

in fact I like this better than node_modules, the venv layout of bin/include/lib is more natural comparing to the odd name of "node_modules" in my opinion, and I don't need npx etc to run commands under node_modules either, all are taken care by venv with its 'activate' script

pip install --user and sudo pip install won't break your venv. But they will break your system Python and any OS commands that depend upon system Python, perhaps including pip and virtualenv themselves, which is incredibly confusing. I've helped both friends and coworkers un-break it, and the symptoms aren't generally obvious. I wrote the patch to pip in Debian to prevent sudo pip install from removing files from Debian packages via upgrading packages. It's a huge pain, it's only worth running if you know exactly what you're doing, and as someone who does know exactly what they're doing I can attest that it's never necessary. After all, you can always just make a virtualenv.

One thing I did at my last job was to make a Nagios alert for machines with files in /usr/local/lib/pythonX.Y/site-packages, indicating that someone had run a sudo pip install, which was super helpful for "why is this machine behaving slightly differently from this other machine which should be identical". We had a supported workflow involving virtualenvs and also we had multiple members of the Debian Python team on staff if you needed systemwide packages, so not only was there always a better solution, there were people to help you with that better solution. :)

Re activate/deactivate, that's a matter of taste but I find it easier to avoid it completely too - see my reply in https://news.ycombinator.com/item?id=20672299 for why. Basically, you get the simple rule of "Never run bare pip" instead of "Remember which pip is your current pip and whether it's the one you meant."

> But they will break your system Python and any OS commands that depend upon system Python

Sudo pip install might on some distros (and I consider this to be a bug on the distro level, not a Python issue) but I've never heard of --user breaking anything

Maybe I'm misremembering, but, isn't the point of pip install --user to get things onto your import path when running the base Python interpreter, just like sudo pip install would (except scoped to your user)? If so, wouldn't installing an incompatible newer version of some library (or worse, a broken version) break system commands that import that library, when that same user is running those commands?

$ source some/directory/bin/activate

And you don't need to keep typing out the full directory name. When done with the environment

$ deactivate

I've grown to dislike activate, because it breaks the simple rule of "never run pip, python, etc., only run your-venv/bin/pip, python, etc.". Now the rule is "Don't run pip, python, etc., unless you've previously run activate and not deactivate" - and it has the complicated special case of "make sure the command exists in your virtualenv." (For instance, it's definitely possible to have a Python 2 virtualenv where pip exists but not pip3, and now if you run pip3 install from "inside" your virtualenv it's global pip! Or you might have a Python 3.6 virtualenv and type python3.7 and wonder where your packages went, or several other scenarios.)

If you have the shell prompt integration to remind you whether you're in a virtualenv or not, it's fine, but I don't always have it, and I find it helpful to manually type out the full directory name (generally with the help of up-arrow or tab...) so I know exactly what I'm running.

for bash/fish it automatically prefix with a (your-venv-name) so it's obvious you're under some venv, not sure about csh but I would assume it will do something similar. looks like venv supports bash/csh/fish only by default however.

This is good advice, but it elucidates the problem. There should be 2 steps (1. list your dependencies, 2. pip install). Not 5.

I agree. I'm optimistic about tools like Poetry https://poetry.eustace.io/docs/basic-usage/ for solving this. Unfortunately Python predates the realization that good packaging tools were a thing that needs to be solved in the core language and not externally (Go, Rust, Node, etc. postdate this realization; C, Perl, Java, etc. also predate it).

The flip side is that decoupling the interpreter/compiler from the build system makes it more possible to write tools like Poetry (and, indeed, virtualenv) that explore new approaches to the problem. At my day job where we do hermetic in-house builds with no internet access and (ideally) everything built from checked-in sources, building C and Java and Python is straightforward, because we can just tell them to use our source tree for dependency discovery and nothing else, and we can set up CFLAGS / CLASSPATH / PYTHONPATH / etc. as needed. Building Go and Rust and Node is much more challenging, because idiomatic use of those languages requires using their build tools, which often want to do things like copy all their dependencies to a subdirectory or grab resources from the internet.

Of course, given that it's Python, there should be one - and preferably only one - obvious way to do it....

>Unfortunately Python predates the realization that good packaging tools were a thing that needs to be solved in the core language and not externally (Go, Rust, Node, etc. postdate this realization; C, Perl, Java, etc. also predate it).

Sure, but it's also a cultural thing. Ruby is nearly as old, and also predates this, but has nowhere near the insanity of Python. The community jumped on bundler and rvm/rbenv/etc super quickly, and rapidly improved them, while the Python community is barely even aware of pip-tools / pipenv AFAICT. Even virtualenv is really only a "real pythoners know" thing, it's rarely mentioned in guides, so newbies screw up their global environment frequently.

Ruby was exotic until someone translated the docs to English, but the whole ecosystem is indeed one reason I love Ruby. I really don't understand why python 2.7 is still a thing 11 years later. Sure, legacy systems, but if I install recently active open source on my machine I wouldn't expect it to use an outdated version of a programming language. Upgrading can't be that hard.

python 2.7 is still a thing because it was announced as the LTS* release over a decade ago.

* nobody called it that, but thats effectively what it meant to say “there will be no python 2.8. Python 2.7 will be supported until T. Python 3.x will be superceded in a year.”