Hacker News Reader

Comment by chuckadams | Hacker News Reader

Python the language is pretty nice. It has its warts, but I make my living in PHP which is practically made of warts. But the python ecosystem still seems to be trying to figure out this whole package management and project setup thing. In most languages I can do some form of `$blub install` where $blub is the language's official package manager or some close equivalent. It's just python that always screams at me that I have to set up and "enter" a virtualenv. I get what venv is for, but it's still a weird hack of hardlinks and relative paths that no other language seems to need, and a clumsy two-step dance of a UX that hasn't improved in like 20 years.

View on HN

I am not sure you get what virtualenvs are: Python is never screaming at you to set up a virtualenv, it must be a particular package recommending use of virtualenv for easy set up without interfering with the rest of your system.

Virtualenv allows you to seamlessly run multiple Python ecosystems simultaneously, even within the same project directory. It's basically primitive containerisation mechanism that predates any actual containerisation systems on Linux.

You do not have to use it, but then you can easily slip into a sort of "DLL hell" (multiple incompatible library versions installed system-wide) with multiple projects — or need to bundle all dependencies within your project directly. None of this is specific to Python, really — any shared library system has the same challenge. How many other systems are there in active use making it as easy to use multiple incompatible versions of shared libraries per project or within the same project?

When in doubt, you can always retreat to the basics in Python world: put packages you need in a path of your choice, and point PYTHONPATH (sys.path) at it.

I thought I'd given sufficient clues that I actually do in fact know what virtualenv is. I was hip-deep into python when it was introduced, and thought it was a clever hack, but it's relying on python being compiled to find its libraries in a path relative to the python binary, so rather than use something sensible like a launcher that sets PYTHONPATH to a local dir like every other language does, it hardlinks the python install into the local dir in order to pretend everything is a system-wide global install. I always figured that hack was a temporary workaround, but it's going on a couple decades now.

As for the screaming, I'm talking about pip, which has the unique property of just refusing to work by default unless you use --user (what most other package managers now call "global") or are in one of these pretend-global environments. Ultimately it's just a paper cut, not a show-stopper, but it points to not just a failure to address the pain points of the package ecosystem, but a seeming refusal to do so.

Usually running pip as root overcame that problem — but it did cause all the other problems virtualenvs were introduced to work around (so I am not suggesting this as the solution).

You are right that there are many problems with packaging in Python, and yet I feel like virtualenvs are the smallest of those.

I believe we also need to compare tech this old to tech from the same era: obviously newer ecosystems had the benefit of hindsight, but how does managing dependencies compare between Perl and Python, for instance?

The biggest problem with Python packaging is — IMO at least — that it is actually attracting so many evolutions and proposals that ot is hard to stay on top of if you do not make Python packaging your core interest.

For managing dependencies in Perl, it was originally a similar story to Python: everything was system-installed, but many people would install things to their home dir and set PERL5LIB in their .bashrc. The cpan client was smart enough to detect and use the home install when writing its initial config, so you could call it a day. Later there was local::lib which fiddled @INC for the use of a project-local directory, and cpanminus defaulted to using it, and then Carton came around which is more or less a clone of Bundler from Ruby, also using local::lib under the covers.

Reply on news.ycombinator.com