Hacker News Reader

Comment by kgwgk | Hacker News Reader

For most people it’s not blatantly obvious that running

  python -m pip install ./totally-legit-package.whl

in a folder will execute a file pip.py in that folder

View on HN

The attack they're describing is even more subtle than that. The problem is that you could have zip.py in your downloads folder, and if pip itself has `import zip` then it will pick that up in preference to the standard library one.

You’re right. Even something like

    jupyter notebook ~/Downloads/anything.ipynb

is a potential risk and may run “modules” in ~/Downloads

It will also execute libc.so and libpcre.so from that folder. That's simply how libraries are loaded on both Linux and Windows, nothing specific to python.

It's really far fetched to go for python or jupyter when there are easier options like system DLL. The downloads directory is fundamentally unsafe, that we can agree on. Thankfully browsers don't let website push files unlike what the author may imply.

I don't think running something like

   /usr/local/bin/notpython ~/Downloads/legit.file

will load libraries or execute code in ~/Downloads

That seems something specific to python/jupyter

But maybe it's more analogous to ask about running ~/Downloads/legit_program (or maybe ...\Downloads\legit_installer.exe). As I said in my sibling comment, this is safe on Linux (if legit_program really is safe) but they're right that on Windows it's dangerous because the program will pick up zlib.dll etc. in the downloads directory. Maybe this is part of the reason for .msi installers: they end up running the MSI installer executable, which is located in some system directory instead.

Sure, running executables in random places may be risky. But with python it’s not just the location of the binary program that matters and running scripts or notebooks stored in random places may also be risky.

Comments up thread note that lua will do the exact same thing, Perl used to do it as well (now changed).

Ruby apparently does not though (the internet shows many asking how to do it). Good for them.

Well, I didn't literally expect it to be unique to python :-)

> It will also execute libc.so and libpcre.so from that folder. That's simply how libraries are loaded on both Linux and Windows

That's true on Windows but NOT on Linux (which I actually find a bit annoying for deploying applications to users). On Linux you could explicitly add "." to your LD_LIBRARY_PATH variable but even then I think it would look in the working directory rather than the executable directory.

And since XP SP2 windows by default searches current directory as one of the last places that it looks into. It is not exactly perfect mitigation but probably works for most instances of this issue.

I think the main concern here is the directory containing the application (especially if it's in the downloads directory), which is always first in the search order on Windows [1]. There was an article in the Old New Thing blog about how this is because the application's directory is expected to be its own safe package, usually a subdirectory of Program Files.

You're talking about the current working directory, which can be different, and you're right it was moved later in the search order in XP SP2 (also mentioned in [1]). I had no idea it was ever searched at all, so thanks for that!

[1] https://docs.microsoft.com/en-us/windows/win32/dlls/dynamic-...

> which I actually find a bit annoying for deploying applications to users

Linux ELF binaries can have a library search path embedded using the -rpath linker option. Inside that search path, '$ORIGIN' refers to the binary location so you can have the Windows behavior if you want to.

Amazing, thanks! I already knew about the embedded -rpath but hadn't heard about $ORIGIN before so I always dismissed it as not being much use. According to this StackOverflow answer [1] you need to use the -z origin switch but I only found that thanks to you pointing me in the right direction. I'll be adding this particular gotcha to my deployed executables immediately!

[1] https://stackoverflow.com/questions/6324131/rpath-origin-not...

If I understand the article correctly, it won't do that unless another condition (misconfigured PYTHONPATH) is also met.

That's not what the article says. The part about PYTHONPATH comes later and points to a different issue.

When you run a script, the location of the script is added to the path.

  # cat > /tmp/test.py
  import sys
  print(sys.path)
  ^D
  # /usr/bin/python /tmp/test.py
  ['/private/tmp', .....]

And when you don't, the current directory is added to the path.

  # cd /tmp
  # /usr/bin/python
  >>> import test
  ['', .....]

The real question is who runs python -m pip when there is a perfectly good /usr/bin/pip?

That's an easy question to answer.

python -m pip is often advised over pip or /usr/bin/pip, because it uses the version of pip associated with that particular major+minor python install.

https://snarky.ca/why-you-should-use-python-m-pip/

Wait, but isn't the point of the article that it will not do that, and rather just run the pip.py in you current working directory?

Only in the (probably unlikely) case that you have a `pip.py` in your CWD. In all other cases, it will execute pip as intended.

Even with the other form, if pip imports anything named like a pyhon module in the dl dir, it will be imported. That's because the current dir is automatically added to the PATH, using -m only add pip.py to the attack surface.

In the off chance that you have one, yes.

But in 99.9999999% of cases it will do what you expect (whereas "pip install x" will be easier to use the wrong one if you have multiple Python versions).

Guilty as charged. Using the system pip in an environment that doesn't use the system python is a recipe for disaster.

So is the common practice of running 'sudo pip install' which can create a world of pain.

I do it all the time.

I do it on Windows using the python executor py. So I'll run something like py -2 or py -3 or even py -3.8 to start the correct version of python when I'm not inside a venv.

Also the same goes for modules. Since I've installed youtube_dl in my global python 3 I'll always run it like py -3 -m youtube_dl and update it using py -3 -m pip install youtube-dl -U. Or to create a new python 2 venv (for whatever reason) py -2 -m virtualenv venv.

for a very long time (I don't use python as a daily driver, so not sure if it's still the case) guidance was to use `python -m pip` and if you tried to run `pip ...` it would explicitlytell you to run `python -m pip`. I regularly install python on clean systems and all of my scripts run `python -m pip`. Should I be changing?

You should always use this form if outside a virtualenv.

Reply on news.ycombinator.com