Hacker News Reader

Comment by johncena33 | Hacker News Reader

I've been programming in Python professionally for more than five years. I consider myself quite a good programmer. I still don't have good grasp on Python's import system. Does anyone else have similar experience?

View on HN

When in doubt, put some more dots in front of your import statements. Or remove them? Maybe I need an extra __init__.py somewhere? Oh I'm importing from a subfolder, what do I need to do to get that work again? I can't remember.

This

I'm a python readability approver at Google and I don't understand how the import system works

Interesting... If someone wants to know more about what a readability approver does: https://www.pullrequest.com/blog/google-code-review-readabil...

To be fair, Google's python avoids ~99% of the complexity of Python's import system by making all imports absolute and doing most things through blaze/bazel.

I’ve only been writing Python for about a year, but I’ve found it much harder to grasp how dependency resolution and imports work than other languages I’ve picked up (JVM, Node, Go, C).

Same. Occasionally I will get into some sort of mess, learn how it works under the hood enough to get myself out, and the promptly forget everything.

And I think that's for the best. I'd much rather have a happy path that I stay on than use some sort of dark magic that nobody who comes after me will understand.

I have >10y python exp. Vanilla import system? I grasp it quite well.

- Entrypoints? Getting there.

- Namespace packages? Ehh. Murky

- site-packages/mypackage.pth - I get it but I don't know why sometimes it appears and other times not

- c extensions? Ehhh.

- .so loading? Kinda magic.

- the confluence of editable installs, namespace packages, foo.pth, PYTHONPATH, sys.path, relative imports, entry points, virtualenvs, LD_LIBRARY_PATH, PATH, shell initialization, setup.py vs pyproject.toml? Um yeah that's some heavy wizardry.

Tbf, you don't need the vast majority of that to be effective in python.

The average python programmer does not really need to deal with pythons import system that much (just be aware of how it does its module loading and that you can conditionally do stuff sometimes with __import__ etc). As someone who has messed around with it a lot (dynamically loading/unloading modules, modifying on the fly etc) I would NOT recommend doing that stuff for anything in production.

I've actually found the Python import documentation [0] to be really approachable. I'd give it a look if this article didn't clear things up.

[0]: https://docs.python.org/3/reference/import.html

I sometimes fall into the trap of using pip to install dependencies and then things break after an os update. That is, my python version has changed from 3.8 to 3.9 and my dependencies are sitting in the wrong directory. I never know if I should use pip and requirements.txt or rely on Ubuntu's packaged versions.

I used to have a rule of, never use pip and only use Ubuntu's/Debian's packaged versions. That works pretty well if you're happy with the packaged versions and you don't need unpackaged libraries.

I now have the rule of, only ever use pip inside a venv. If your venv is more than a little bit complex, write a requirements.txt file so you can generate it. So it's something like

    $ cat > requirements.txt << EOF
    tensorflow
    Django==3.2.5
    cryptography
    EOF
    $ echo venv > .gitignore
    $ python3 -m venv venv
    $ venv/bin/pip install -r requirements.txt
    $ venv/bin/python

or, if you prefer,

    $ . venv/bin/activate
    (venv)$ pip install -r requirements.txt
    (venv)$ python

Then when your Python version changes, or you get confused about what's installed, or whatever, you can just blow away the entire venv and recreate it:

    $ rm -r venv
    $ python3 -m venv venv
    $ venv/bin/pip install -r requirements.txt

and you're in a known-good place.

Either of these rules works fine. The thing that works poorly is using pip install outside of a venv (with or without root).

For me the rule is to always use pipenv locally and pip + requirements.txt (generated by pipenv) for production (in docker container usually). No complaints.

Just pip install again under new interpreter. Use distro versions if a script will primarily run there, and ok if an older version.

same! I've always been shielded from it with Django's conventions. (the ecosystem I mainly work in). I used a lot of '.' and '..' imports but I think something changed in python3 that made that strategy a lot less forgiving... now I _really_ should read the entirety of this article!

Always use absolute imports, do not use relative imports. Solves most problems. Also is recommended by pep8.

Skip the relative imports in the article. No need to read it entirely anymore ;-)

I thought not knowing that made me a noob.

Reply on news.ycombinator.com