The return of lazy imports for Python | Hacker News Reader

The return of lazy imports for Python

lwn.net

113 pointsmfiguiere88 comments

I really hope a solution can be found and python can get true lazy imports (without doing the manual thing). I work in the ML/AI space and manual lazy imports are basically mandatory unless you want several seconds to see usage text from `footool --help`. Why "pay" for executing code you don't use?

Manual lazy import meaning:

    def uses_foo(x):
        import foo
        return foo.bar(x)

Manual lazy import sucks because:

- it's just ugly, I like imports all at the top so I can see all the deps

- bad for static analysis

- performance hit every time the function is called

Eschewing lazy imports has several problems:

- you always pay the execution cost, even if you don't use it

- also bad for static analysis and testing, since you have to eat the import time even if the code block you want to test doesn't execute the expensive path

- sometimes you need lazy import to avoid circular import errors

It's too bad that the main impediment to this is existing code which relies on side effects. Import-time side effects are an absolute pain in the ass. Avoid it at all costs.

Given that modern ML libs can bring hundreds of megabytes of binaries with them, this is a great point. As for static analysis in Python, I am not sure it can get much worse :) The language is essentially built around the idea of making everything absolutely dynamic

Typing in python is pretty thorough nowadays, you’ve got overloads, literal types, strict null checking, unions

It’s a more advanced type system than Go and Java

Typing is very surface level though and doesn’t negate what the person you’re replying to is saying: Python is über dynamic.

It’s very trivial to have something pass the type hinting checks, but be completely different at time of use. Even if you hold on to a single instance of an object, it’s easy for any thing to monkey patch it.

> - performance hit every time the function is called

Python modules are always singletons, regardless of where they are imported. "import foo" inside a function will only import the module once (=global effect) but bind the name every time (=local, but cheap, effect).

But you're still getting the perf hit of calling a function and checking if the module is already loaded

It’s a fairly simple dict lookup. The same lookup would happen when you use something from that module, so it’s fairly insignificant in the grand scheme.

Besides, it’s Python. It’s not going to be super fast anyway. That extra check is never going to show up on a perf trace.

Yeah well there's never been a line of Python that's free

With one exception -- whatever module is executing is named `__main__`, and if something else imports it by it's usual module name you'll get a duplicate copy.

Example:

  $ cat spam.py
  import sys
  print(f'__name__ = {__name__!r}')
  for (name, mod) in sys.modules.items():
      try:
          if 'spam' not in (mod.__file__ or ''):
              continue
      except AttributeError:
          continue
      print(f'sys.modules[{name!r}] = {mod!r} @ 0x{id(mod):x}')
  import spam
  
  $ python3 -m spam
  __name__ = '__main__'
  sys.modules['__main__'] = <module 'spam' from '/home/jwilk/spam.py'> @ 0xf7d86ed8
  __name__ = 'spam'
  sys.modules['__main__'] = <module 'spam' from '/home/jwilk/spam.py'> @ 0xf7d86ed8
  sys.modules['spam'] = <module 'spam' from '/home/jwilk/spam.py'> @ 0xf7caef78

More generally, Python is happy to import the same file multiple times as long as the module name is different.

For example, if there's eggs/bacon/spam.py and you add both "eggs" and "eggs/bacon" to sys.path, you will have two different modules imported after "import bacon.spam, spam".

One approach I've done, to try and reduce the performance hit, is:

    def uses_foo(x):
        global uses_foo
        import foo
        uses_foo = foo.bar
        return foo.bar(x)

While that performance suckage improves, the other suckages get worse.

Doubtful lazy imports would’ve helped at all… I joined a project where almost every import statement has side effects, some of which took multiple minutes to read things into memory etc.

Tried some poor-man’s debugging and never hit a breakpoint on the first significant line of code… took a while to figure out as it was my first Python project.

It almost feels like Python needs a scripting and non-scripting mode, or some kind of warning logging “you did everything wrong”

Python is a very good scripting language, so good that people sometimes mistake it for application language.

All of Python, Perl (!), Ruby, Tcl have been application languages at one time or another. Python evangelists, reel it in a bit, please.

I have yet to understand how to clearly separate scripts from applications.

For me it is AOT compilation and packaging. An application language should be able to pull in most of its dependencies and runtime and be packaged into a reasonably sized and performant binary. The reason why scripting languages (that are often dynamic) struggle with this is that too many of their dependencies have hotpaths written in third party languages like C which makes it hard to do easy cross compilation and linking. Some programming languages come from a culture that eschews AOT compilation. It is difficult to compile large dynamic languages with Lisp heritages (CL, Julia, Ruby etc., Scheme is an exception to this however). These languages always resort to using a stateful VM/interpreter snapshot in lieu of true AOT packaging (or even Java/.NET style lowered bytecode binaries). The existence of eval makes many things trivial but at the same time complicates deployment tremendously.

Personally I am betting on JavaScript being the first dynamic and JITted (or incrementally compiled) language that can attain the AOT compilation without massively bloated binaries.

So wait, then C# wasn't an application language before there was a way to package AOT binaries to deploy directly to users?

It is, because the runtime is bundled with the OS (similar its counterpart the JVM is also extremely widely deployed) and its output is significantly lowered. C.f.

> (or even Java/.NET style lowered bytecode binaries).

Re. AOT, as I said in my sibling comment, there's a counterexample -- Julia. It's AOT compiled, but dynamically typed. And it falls in scripting camp, IMHO.

Julia only does partial AOT, it still bundles the sysimage (~100MB+ of runtime, what I refer to in my above comment as the VM snapshot) c.f. https://docs.juliahub.com/PackageCompiler/MMV8C/1.2.7/devdoc...

Traditional AOT like you see in C/C++/Rust/Go/Zig would be able to treeshake and eliminate redundant codepaths, the binaries are all fairly small with minimal startup overhead.

Makes sense, thanks, I don't know much about Julia.

Another counterexample to AOT compilation being a feature of application languages is Java and C# -- both are clearly application languages, with strong focus and presence there, but both are interpreted. Although I can argue that it depends on the terminology regarding "compile", whether transpilation to .class bytecode can be called compilation (and everyone do call it "compilation", but if it really was, there would be no point in real AOT compilers for Java/C#).

Mostly it's original focus of the language and ecosystem developers, IMO.

Although I found one feature to very strongly correlate -- static typing. For example, Julia is AOT compiled, but still dynamically typed. And coincidentally, it was designed for scientists, who need scripting much more.

Another observation is how easy it is to write unit-tests (for canonically written code). Overloading "import" statement doesn't help there at all.

I added lazy loading by default to my workplace's monolith in Node.js. We relied on a couple of imports with side effects, which no longer worked because no one was accessing the modules that ran the side effects, and that was the trigger for actually loading the module.

Our fix for this was to relegate stateful imports to files with "bootstrap" in the name, which the lazy loader stub would allow to be eagerly imported. Moreover, any imports listed in such "bootstrap" files would then be eagerly imported. But that's it (at least wrt the code belonging to our codebase); no one else, not even the children of the bootstrapping modules, were exempted from lazy loading.

This allowed for a centralization of all the stateful import effects. If you tried to write stateful code outside of bootstrapping, it simply wouldn't run. (You could hypothetically hack around this. But it would be rather obvious, and I'm perfectly ready to revert pull requests that try.)

Maybe you folks could try some variation of this in Python.

Import side effects are the devil and should be killed with fire wherever they appear.

Amen

The problem is that side effects are really broad and include things that are idiomatic Python.

    # jobs.py
    @scheduler.register_job(when=“daily”)
    def batch_job(): …

This adds the job to the global scheduler.

    # models.py

    class Model(DBModel):
      def on_change():
        # do stuff

This adds the model to the ORM’s list of modules via a metaclasss and registers its hooks.

    # plugin.py

    class SomePlugin(PluginBase): …

Same thing. The code to load plugins like this is just importing the module and letting the metaclass do the work.

Not a word though about the elephant in the room: circular imports.

It is absurdly easy in Python to end up with a circular import situtation, where no real circular dependency exists. E.g. you can't have A.a1 -> B.b1 and B.b2 -> A.a2. So, you are forced to layout your code in some quite awkward ways.

Isn't having circular dependencies more awkward? Conceptually, it makes things more intertwined when instead you can build a better and more separated architecture.

Is there an elegant way to import type hints without circular imports?

Something like this does the trick:

if TYPE_CHECKING: import WhateverClass

https://docs.python.org/3/library/typing.html#typing.TYPE_CH...

I’m a fan of having a single state, used for everything. Splitting the code up into two states, one for the linter and one for the execution, seems like a recipe for incorrectness and confusion. I would hate to refactor something like that.

The issue is that sometimes a function can take a type that is an optional dependency, so you don't want to import it unless you are type checking.

(And some types are defined in the typeshed so only exist to be imported during type checking; eg the type checker lib itself is a dependency in this case)

It's hardly elegant, though.

Yeah if you’re like just put what is essentially

    if False:
        import blah

unironically as good design and more than a necessary evil until a long-term solution emerges then we’ve jumped the shark.

Type hints can also be strings, which at least PyCharm resolves as if they were real type references

    from typing import List
    class Alpha:
        @staticmethod
        def doit(b: "Beta") -> List["Beta"]:
            return [b]
    class Beta:
        @staticmethod
        def doit(a: "Alpha") -> List["Alpha"]:
            return [a]

This only works if they're in the same file, or Alpha imports Beta later in the file and vice versa. Once you split these into separate files (say for example adding a gamma.py file and Gamma class that uses Alpha and Beta) you get Unresolved reference 'Alpha' and Unresolved reference 'Beta', and typechecking calls to Alpha.doit and Beta.doit from Gamma does not work.

I thought it also accepted fully qualified type names but I just tried it and you're right. Relevant to this submission's title, if your packages are side-effect free, just importing the namespace then allows referencing the type names without touching the "actual" type

  import gamma
  def doit(bar: "gamma.Gamma"): ...

In Java my answer to circular deps is the introduction of an interface that the concrete types can implement but then breaks the cycle

The only way to really fix that is to disallow code running outside of functions, IE, basically a whole new language.

Yeah the module thing is my biggest irk with Python - merely importing a module for symbols executes all the top level code.

Personally, unless it has explicit "lazy" syntax I kind of hate the idea. One thing I always liked about python was how predictable and simple module imports are.

I think such a mechanism already exists. You can just use the functional import syntax inside of a block. However, I think lazy imports could be ok so long as the language can show that a particular module is side effect free (i.e. no globals aside from something like constexpr).

> You can just use the functional import syntax inside of a block.

You don't even need functional import syntax, but as TFA notes this comes at a cost as it has to invoke the entire import machinery, which it can only skip to an extent (once a module is loaded and cached) as import hooks can have odd behaviours.

I wouldn’t call different behaviour whether there’s __init__.py predictable nor simple. Also relative imports not working in an interactive session.

The only argument PEP 690 has is A) [performance on startup] or B) [when import functions is not used] in main body of the code.

For B), easy enough to run one of many linters to detect this case and have people write less bad code.

A) is way more subjective and can be fixed in many ways.

With the many more Python coders these days with less coding experience, personal feeling is please stop throwing these production issue causing features in that I have to fix. Glad the PEP is rejected.

Old programmer wisdom is to load all your configs and assumptions as early as possible to eliminate a whole space of problems with your code, making faster and easier to read/reason about later.

I’m curious about the other ways to fix startup performance.

I’ve seen a moderately sized (~300k LoC) Python CLI project that had a horrendous, anger-inducing startup time until they switched to the lazy import approach basically described/standardized by PEP 690 and the improvement was massive.

B sounds great but too much of the pydata ecosystem breaks here, like `class MyNeuralNet(nn.Module)`

For sure, pydata ecosystem is one of the worst offenders. Yeah, the libraries make it easy to do that 1 off model -- shove it in production and watch the next 3-6 months be a pagerduty fest for both the model writer and ops/backend people.

It doesn't have to be like that though -- look at geohotz's tinygrad library for exampe: well tested, well written, and can do most of the things the bigger ecosystem libraries do.

The goal of lang features like these is to automate the optimization especially when it is hard to do at scale without it. Scenarios that start with "If only everyone..." are great candidates.

Conversely, agreed, it's not THAT hard to convert an individual project, just being unable to handle third-party, so import-shaming some top projects can likely go far. Features that enable framework providers to empower others are great... And none is needed here. It's messy to push through type checking, but doable.

Feel like we are reliving when js got a more static module system. A lot of kicking and screaming, and still issues, but a lot of good came out of that.

Library authors can make their modules initialize lazily. It's additional code complexity, but it's probably worth it in certain cases. At one point, I was on a team developing a complex extension module that would take several seconds to import since it produced a gigantic shared lib. Sure, that probably could've been made more modular, but lazy initialization was easier.

https://docs.python.org/3/library/importlib.html#implementin...

I really appreciate both the good writeup here and the fact that so many people are thinking through changes so carefully.

I didn't understand this when it first came up and I still don't. If you want to defer your imports than wait until it's needed. It might be useful if you need to load a behemoth of a module for some rarely used part of a CLI tool. Otherwise, an X ms load at startup is hardly any different than the same X ms load in the middle of the execution. And on a server it's actually worse.

It's hugely different. The issue occurs when you have something which does not need to execute in every context. Let's day you have a CLI which does some machine learning with several models. You don't want to execute loading every model every time you start the app.

The most egregious is when you just want to display the help/usage text and not actually execute any code at all. Instead, you have to either manually lazy import (what I usually do now), or eat huge startup costs each time you screw up the command syntax.

Every single CLI tool gains from this. Most Python CLI tools have terrible UX since stuff that's as simple as running it with `--help` or `--version` requires the VM to go crawling around everywhere on your disk and executing bytecode.

That's a design problem, imo, not a Python problem.

I get that, but that's poor design on the tool's part, not Python's problem. Hide your imports behind functions, don't import at the global level, unless it's needed globally.

It may not feel very clean but one solution for slow imports is to write your own module that imports fast. To give an example: I had a some script whose startup was slowed down by importing numpy for one small (but important) thing.

Replaced it with a 20 line c extension that imported basically instantly.

I wonder if it's worth adding a new `import lazy` statement for the next major release? I was about to suggest `import` being lazy by default and introducing an `import strict`, but that would fuel another 2->3 style nightmare.

The people behind the xonsh shell has already implemented lazy loading of things:https://github.com/xonsh/lazyasd

Xonsh shell is amazing.

Lazy imports don't really seem that useful. The only time I've found them useful (in a Ruby project) was for unit tests/local development where only a small subset of the application is loaded at a time. Anything long running you generally want the predictability of loading everything up front. For command line utilities, it seems like you're going to need to load the module at some point or another regardless (if you're actually using it) so I'm not sure how you'd see a gain unless there's some async/multi thread hacks.

In really large projects (e.g. SciPy as mentioned in the article), lazy imports make sense. Especially with the popularity of decorators, importing a file without any apparent module-level code will actually need to run a nontrivial amount of code. Multiply that by a few thousand files in a library with a tree of "import * from ..." and you're looking at perhaps seconds of startup time. Lazy importing can short-circuit that, but still make symbols available for ease of use.

Numpy and Matplotlib were quite slow to import also, I haven’t timed them recently though.

Some package developers don't want their users to have the two step process of "import" than "use." NumPy imports 137 modules with "import numpy", of which 94 are specifically in the NumPy hierarchy:

  >>> import sys
  >>> len(sys.modules)
  83
  >>> import numpy as np
  >>> len(sys.modules)
  220
  >>> sum(1 for k in sys.modules if "numpy" in k)
  94

so people can write a one-liner like:

  >>> np.polynomial.chebyshev.Chebyshev([0,1,3])(np.linspace(-1.0, 1.0, 5))
  array([ 2., -2., -3., -1.,  4.])

without having to import np.polynomial.chebyshev.Chebyshev first.

This API design requires importing most of NumPy at startup, which has a cost they didn't consider so important because their users are primarily doing long-term computing and notebook-style development, where startup cost is relatively small.

I've complained about this because I live in the short-lived program world, where it's annoying to have a 0.1 second import overhead if I only need one function from NumPy:

  py310% time python -c 'pass'
  0.025u 0.006s 0:00.03 66.6% 0+0k 0+0io 0pf+0w
  py310% time python -c 'import numpy'
  0.142u 0.292s 0:00.14 307.1% 0+0k 0+0io 0pf+0w

As I understand it, SciPy wants a similar API design goal, but has a lot more packages. They've developed lazy imports to try to have the best of both worlds.

> For command line utilities, it seems like you're going to need to load the module at some point or another regardless (if you're actually using it)

Thing is, you might not actually use it. If the command-line tool uses subcommands, each different subcommand might need only a subset of the full set of packages.

Perhaps only one of the subcommands uses NumPy, while for 95% of uses, NuPy isn't used at all.

As the discussion for this feature points out, this can be addressed by only importing when needed. (One of the reasons I've started using click over argparse is click does more of this separation for me.) However, it's somewhat fragile, in that it's easy to add an rarely-needed expensive import at top-level without noticing it, and requires some non-standard tooling to detect issues, like the non-predictability you mentioned.

I personally want something like the lazy-/auto- importer in my package, so I can reduce the two step process. My last package released used module-level getattr functions, which gets me mostly there, except for notebook auto-completion of the lazy wrappers. (It works in the command-line shell though.)

I can't import everything on startup because parts of my package depend on third-party packages, which might not be installed. I instead want to raise an ImportError when those lazy objects are accessed. Plus, one of the third-party packages is through a Python/Java bridge, which has its own startup costs that I want to avoid.

You could do the numpy style API lazily. They would just need to each API as an object that does the imports dynamically.

... yes. The PEP describes existing solutions, followed by a proposal for "a more general and comprehensive solution for lazy imports that can encompass all of the above use cases and does not impose detectable overhead in real-world use", to quote the PEP at https://peps.python.org/pep-0690/ .

I get an impression that regardless of topic, it’s difficult for any decisions to be made for the future of Python? The discussion seems to always revolve around “what if?”s in not the most collaborative fashion. I wonder if what most languages need are less experts in computer science or language theory or anything technical, and more folks that can do facilitation.

Why do you think managers take over everything? Regardless of topic, any sufficiently large problem eventually becomes primarily a coordination problem

Personally, I’d much rather languages be designed from mathematical foundations and/or very careful theory.

There certainly are languages like that, but I think you'll find there are tradeoffs to consider. Especially in commercial software.

The ones I’m familiar with (clojure and haskell) are a joy to work with. These languages have conquered my heart!

Python decision-making is rather conservative, preferring deep exploration of implications, because its a big, established language with a lot of existing use to support, and because of the 2->3 experience.

I don’t think lack of facilitation skill is an issue; its a deliberate policy choice.

Is there any production ready language that isn't conservative with its decision making?

> Is there any production ready language that isn't conservative with its decision making?

There's variations of degree, but probably not. Part of being production-ready is stability.

The what if is a valid question I think. In my little corner of the universe, my boss is genuinely ( and the more I think about it, reasonably ) worried about introducing more dependency on Python in our daily work.

The are a lot of reasons not to introduce it, but 'what ifs' at a company like ours could be devastating. I still think proper precautions can be taken, but it is harder for me to say that I would just say yes if I was in his shoes.

Isn't it a good thing that decisions which affect billions of devices are taken carefully and deliberately rather than rushed? Is there an expectation that major decisions for the future of one of the oldest and most widely deployed programming languages in the world should be easy?

The import code in Python is a mess, probably made slower since the advent of importlib.

You need developers who care about fast, clean code to fix the issue. Those kind of developers usually don't fare well in the Python swamp, so it won't happen.

Might not grasp the full context here, but it’s trivial to lazily import modules in your own code. I know every beginners guide will advice you not to do that, but that’s just because it’s an easy footgun for new programmers. If you have some cli tool that only needs scipy for certain sub commands you can just move it to those subcommand calls so it’s loaded when needed instead of up front.

I usually only import argparse inside my ‘if __name__ == “__main__”’ stanza.

That's not really the issue with argparse and subcommands.

argparse with subcommands generally requires specifying all of the options for all of the subcommands, even if you only want one subcommand.

These in turn may require importing subcommand-specific modules, to handle things like the right 'type' handler in an an add_argument() parameter. This callback function might, depending on the input value, select one from a dozen different additional packages.

It's possible to avoid this, by deferring argument->type processing until later, and having a single large module containing all of the help strings and epilogs, though this will separate your argparse code from your subcommand code, and in general make things more complicated. I did this for a while.

Alternatively, you can create your own subcommand dispatch system using an nargs="?" to get the subcommand and an nargs=argparse.REMAINDER to capture the rest of the flags, to pass to a new ArgumentParser, and develop a top-level --help replacements. I tried this too.

I've since decided to use click, which does a better job at compartmentalizing at least this level of subcommand imports.

I've written several large CLI tools with Python and always ended up doing something in this vein. --help - and invocation errors - just take too long otherwise.

As long as you eagerly check if it exists. Don’t wait until part way through your program to discover dependency issues.

There is a downside: manually doing so means your import occurs every time you call that function. This would avoid that by only importing once lazily.

it's an extra function call, but module imports are cached so you're not incurring the actual import cost.

You can say

mylib = None

in the global scope and then

global mylib

if mylib is None:

   import mylib

in your function to avoid the extra function call.

Isn't `is` a function? A cheap function, but still a python function

"is" is an operator, and better yet, it's not an overloadable operator, so it's about the fastest thing you can do to two values in Python.

Why though? import already does the similar check internally. Your global check might actually be more inefficient because it would need to check more dicts instead.

E.g first the local dict, then the global and then the modules dict, instead of just the global+modules

Imports only happen once.

import mod

is the equivalent of

mod = sys.modules.get(name)

if not mod:

    mod = sys.modules[name] = load_package(name)

It’s really low cost, as long as you’re not doing it in a hot loop, it’ll be very low to no impact.

That trivial way only lazily shallow imports. I don’t see a good way to do a lazy deep import. A lot of libraries I import, then transitively import hundreds or more of other files. The file I import I may only need a small subset of those transitive imports. The lazy import pep would have meant that whenever the import was finally executed, the imports in that file are also lazy and only done if needed.

Reply on news.ycombinator.com