Hacker News Reader

Comment by matmann2001 | Hacker News Reader

matmann2001

This is a solved problem and the solution is virtualenv. Learning how to use venv's would have likely taken less time than the tinkering described in this post. If you're going to regularly develop with python and use 2nd/3rd party code, it's worth knowing.

View on HN

The author uses Debian OS packages and he's talking about installing one unpackaged dependency. He's relying on what's packaged in OS and is tested with other packages. He clearly doesn't want to get random dump of dependencies on every installation (given you leave version empty in requirements.txt). Basically, he's relying on Debian for QA of packages. I do the same. I don't need to track versions of gazillion of dependencies for every little Python thing I write. I just install it in the OS and use it from anywhere else.

You can easily combine system packages with additional packages installed in a venv:

  $ python3 -m venv --system-site-packages my_overlay_venv
  $ ./my_overlay_venv/bin/python -m pip install whatever

Agreed - I was a virtualenv holdout until recently and even though it's a horrible solution with replication of junk everywhere it's the way to go.

Deduplication should happen at the filesystem level.

Databases do it, Git does it, ZFS does it.

Ok the first two examples aren't filesystems but they can store files, so not totally irrelevant?

Yeah, agree kind of. But having to resort to duplicating binaries over and over is a sign something isn't right. Not that it's an easy thing to solve.

It depends, binaries are just data. You reference that data with a path on the filesystem. In a programming language it would be assigning a value to a variable. For static values, the compiler/interpreter can optimize and do deduplication. And you expect it to do so because you don't want a global variable holding the value 1 to do the optimization yourself.

Relying on abstractions to simplify the architecture is a sign something might be right. Simply because simple things are less error-prone IMHO.

If you've just now taken the venv plunge wait until you learn about conda. Python is more or less a shitshow when it comes for packaging and it's the reason I'm drifting away from it little by little

You should have seen Perl packaging.

but but CPAN usually worked and didn't throw away an entire previous language in favor of a new and entirely incompatible rewrite!

... oh, wait.

The author explicitly discusses this option and explains why he didn't take it.

Reply on news.ycombinator.com