Hacker News Reader

Comment by isitmadeofglass | Hacker News Reader

Might not grasp the full context here, but it’s trivial to lazily import modules in your own code. I know every beginners guide will advice you not to do that, but that’s just because it’s an easy footgun for new programmers. If you have some cli tool that only needs scipy for certain sub commands you can just move it to those subcommand calls so it’s loaded when needed instead of up front.

View on HN

That trivial way only lazily shallow imports. I don’t see a good way to do a lazy deep import. A lot of libraries I import, then transitively import hundreds or more of other files. The file I import I may only need a small subset of those transitive imports. The lazy import pep would have meant that whenever the import was finally executed, the imports in that file are also lazy and only done if needed.

I usually only import argparse inside my ‘if __name__ == “__main__”’ stanza.

That's not really the issue with argparse and subcommands.

argparse with subcommands generally requires specifying all of the options for all of the subcommands, even if you only want one subcommand.

These in turn may require importing subcommand-specific modules, to handle things like the right 'type' handler in an an add_argument() parameter. This callback function might, depending on the input value, select one from a dozen different additional packages.

It's possible to avoid this, by deferring argument->type processing until later, and having a single large module containing all of the help strings and epilogs, though this will separate your argparse code from your subcommand code, and in general make things more complicated. I did this for a while.

Alternatively, you can create your own subcommand dispatch system using an nargs="?" to get the subcommand and an nargs=argparse.REMAINDER to capture the rest of the flags, to pass to a new ArgumentParser, and develop a top-level --help replacements. I tried this too.

I've since decided to use click, which does a better job at compartmentalizing at least this level of subcommand imports.

I've written several large CLI tools with Python and always ended up doing something in this vein. --help - and invocation errors - just take too long otherwise.

As long as you eagerly check if it exists. Don’t wait until part way through your program to discover dependency issues.

There is a downside: manually doing so means your import occurs every time you call that function. This would avoid that by only importing once lazily.

it's an extra function call, but module imports are cached so you're not incurring the actual import cost.

You can say

mylib = None

in the global scope and then

global mylib

if mylib is None:

   import mylib

in your function to avoid the extra function call.

Isn't `is` a function? A cheap function, but still a python function

"is" is an operator, and better yet, it's not an overloadable operator, so it's about the fastest thing you can do to two values in Python.

Why though? import already does the similar check internally. Your global check might actually be more inefficient because it would need to check more dicts instead.

E.g first the local dict, then the global and then the modules dict, instead of just the global+modules

Imports only happen once.

import mod

is the equivalent of

mod = sys.modules.get(name)

if not mod:

    mod = sys.modules[name] = load_package(name)

It’s really low cost, as long as you’re not doing it in a hot loop, it’ll be very low to no impact.

Reply on news.ycombinator.com