Hacker News Reader

Comment by Galanwe | Hacker News Reader

I don't really get what this is supposed to solve.

- `virtualenv` works by creating an overlay of the `site-packages` in `$VIRTUALENV`, and change the shebang of scripts to use absolute path of python in the overlay. That way, not only do you get dependencies isolation, but you can also call directly your programs from outside the virtualenv and it will work.

What you propose is both bad practice (relative path in `$PYTHONPATH`) and rather annoying (you need to be in a specific directory to run everything). Trying to automate that will sure end up in pure hell!

- As in "you need to `cd` to _this_ directory then run _that_

- Or "WTF it worked on my machine when I was in this directory but now it doesn't"

View on HN

Well as we mentioned, it's supposed to solve a situation where you don't want to use all the overhead of virtualenv.

The things you care about may be different from what others care about: I'm guessing don't care about Windows compatibility or you wouldn't talk about shebangs, and others may not find only running the scripts from the project root an issue (I don't for example).

Why is a relative path in the PYTHONPATH necessarily bad practice? It's not a rhetorical question, I am genuinely curious.

If you're happy with virtualenv then continue using it - all we have done is propose an alternative.

I'm curious, what sort of overhead are you seeing with virtualenv that leads you to use this/any alternative? I've used virtualenv + virtualenvwrapper on some seriously underpowered and low-on-space machines, and have really not noticed enough overhead to even make a blip.

I'm pretty certain virtualenv just changes some environment variables.

I found a writeup [1] on how to write your own virtualenv. It doesn't look like there's much overhead there.

I suppose one could justify not using virtualenv if they're on a system with limited resources or network access. I've known people who spent hours building a compiling python 2.7 from source on embedded systems because they rolled their own distro, and in that situation it would make sense to not want to mess with virtualenv if you just spent hours getting pip working.

1: https://www.recurse.com/blog/14-there-is-no-magic-virtualenv...

My impression is that the benefit of this approach is that it fits the mental model of NPM and Bower. Being able to manage packages across your stack using the same kinds of steps feels... organized. The relative path issue, however, needs some more consideration.

Yeah I think for me it's more about the intellectual overhead than performance.

With this you just have one tool, and it's just easier for me to understand what's going on compared with virtualenv.

Virtualenv is really not complicated. I think your time would be better spent learning how it works than just hacking some crappy workarounds to skip the problem.

It can be a pain when trying to distribute software to users (non-python programmers). What I usually end up having to do is distribute virtualenv with my code, and include a simple Makefile/build script to setup the venv. Invariably someone tries to then move that directory and everything ends up getting screwed up until I can tell them to run "make clean all".

I'm not too sure that this would end up being any easier, but maybe...

What I'm doing with streetsign[1] is including a download virtualenv step in the setup script[2]. The virtualenv gets set up in `.virtualenv/`, so most users won't even see it.

[1] http://streetsign.org.uk/ [2] https://bitbucket.org/dfairhead/streetsign-server/src/5b359a...

http://nuitka.net/

Does that work with Cython compiled dependencies?

You can build debian packages that way; dh-virtualenv (https://github.com/spotify/dh-virtualenv) makes it quite easy to pack a full virtualenv into a .deb.

A cross-platform solution that's not very different is just to invoke a virtualenv'd script with the version of python you want:

$ /path/to/.virtualenv/my_project/bin/python SomeScript.py

i remember reading virtualenv is like the simplest little program. what overhead are you talking about?

> Why is a relative path in the PYTHONPATH necessarily bad practice? It's not a rhetorical question, I am genuinely curious.

Among other things, what happens if your Python code changes its working directory, then subsequently tries to import a module? Consequences range from "doesn't work" to "introduces a security vulnerability by running arbitrary Python code from an untrusted directory".

Actually I don't believe that would happen. I think the relative PYTHONPATH gets expanded at startup and becomes an absolute path. Subsequent changes to the working directory wouldn't affect it.

That seems correct:

  /tmp $ PYTHONPATH='./test' /usr/bin/python -c 'import sys; print sys.path[:3]'                                                                                                                               
  ['', '/private/tmp/test', '/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python27.zip']

Good to know, thanks.

It's still odd for software to only work when launched from a specific directory, but not necessarily a critical security vulnerability then.

How is npm a model for anything? Just do a

  find . -type d -name node_modules | wc -l

from any reasonably large project's root directory -- the results will defy your sense of parsimony.

Packages which require other modules pegged to a specific version will install that package within their own node_modules directory, when they are already deep within a subdirectory of another node_modules directory. If two packages both require bar@X.Y.Z, they will both install that bar package in their own node_modules directory.

It is literally the most naive way to write a package manager.

What exactly is the overhead of a virtualenv? A few symlinks and a script which adds a few variables to your environment? Your calculation of what is and isn't complex is exactly backwards from the traditional Python perspective.

As a Pythonista, I can't help but object to the homebrew/npm school of roll-your-own package management. It eschews parsimony, deliberate architecture planning, and learning from the mistakes of other projects in favor of aesthetics and (sorry, I can't think of another name for this) hipster-DIYism.

Yes I have sometimes wondered about that. In fact another aspect of npm which I don't get is if you have multiple "instances" of a library (e.g. different versions, or even two copies of the same version) then any library global variables will not be shared across those instances.

But I don't know enough about JS development to say whether this really happens and/or if it is an issue (some pretty cool stuff has been done with JS so I guess it's possible to live with it).

BTW I share your dislike of hipsters, though I am a bit jealous those moustaches are just so cool.

> Why is a relative path in the PYTHONPATH necessarily bad practice? It's not a rhetorical question, I am genuinely curious.

Let's say you reimplement ls in Python. You then use your shiny new program to have a look at some downloaded files:

  $ ../utils/ls.py 
  ['hello.txt', 'os.py']

So far so good. Now let's try it with PYTHONPATH set to include the current directory:

  $ PYTHONPATH="." ../utils/ls.py 
  Boom! Your box is mine!
  ['no', 'files', 'for', 'you']

A caveat of the approach is that you must run the script from the project's folder.

This is apparently an unacceptable limitation for some people, judging by the responses to the article - however for others it's not.

Reply on news.ycombinator.com