Python Type Hints – How to Handle Optional Imports | Hacker News Reader

Python Type Hints – How to Handle Optional Imports

adamj.eu

47 pointsrbanffy30 comments

I swear by type hinting in Python, but it is pretty frustrating how many common patterns don’t work or require very convoluted use of typing.

It’s an uphill battle to convert a team over to using hinting because of how awkward things can get and how easy it is to just pretend the feature doesn’t exist.

The problem is that people try to turn type hints into static types. I got rid of mypy from all of my projects as it provided no value and just created this constant battle of "making it work" - a collosal time waste. As this blog illustrates - so much extra effort for something so trivial and unimportant - this whole shebang goes against the spirit of Python.

In general I've yet to see what these type check systems (be it mypy, pyright or any other) offer over proper tests. Type hints are there for developers not the machine.

I'm going to hard disagree with you here. Type hints are a key tool in working on a large Python project in a large team. Tests are, of course, necessary. But type hints allow Python (and all its benefits) to be used in industrial environments where it otherwise would be intractable.

It sounded to me like they agree with you -- type hints are valuable -- just with the caveat that satisfying the tooling around them is a waste of time when most of the benefits can be had with minimal effort writing type hints explicitly for the consumption of other developers.

E.g., when I write numerical python code I'll often just use a string like `"(n_items, n_dims)"` as the type hint, or if the only salient detail of a signature is that some dimensions match I might even write something as simple as `foo(bar: "(a, b)", baz: "(b, c)")`. Ofttimes that's the only data a developer really cares about in that programming context, so a more complete type hint (one designed for mypy and the rest of the tooling) would just take longer to read and understand.

exactly! What I meant is that returns on mypy compliance investments are extremely poor. For most projects it makes no sense to invest so much time playing this "hack this hint" games but for big projects every single digit% improvement counts.

Most projects don't need it and I can't help but facepalm when I see small projects have half of their commits describing typing compliance battles just because mypy came with "yet another python project boilerplate" or pressure from the industry.

> In general I've yet to see what these type check systems (be it mypy, pyright or any other) offer over proper tests. Type hints are there for developers not the machine.

In a language with proper first-class typing, the advantage is that they're included in your automated refactoring rather than needing manual updates. But yeah I've never found optional type systems to be any use.

> people try to turn type hints into static types

The only way for type hints not to be static types is to not run the checker before deploying.

Static typing means that the types of variables and their values are statically checked (hence the name) before the execution; in most statically typed languages that happens at compile time. There is no static check at runtime, and it's perfectly possible to send an incorrect type to a dynamically loaded library even in compiled languages (which usually, but not necessarily, results in crashing he program).

Python sucks, type hints border idiocy. If you can't trust the hint to actually be correct and still have to check for yourself, they become pointless.

If you want to see where static typing really shines, try writing some Haskell. It's an absolute delight.

What do you mean you can't trust them? If you write a type hint in your code and then run mypy on it, it will tell you whether the type is being violated. And, PyCharm has a type checker built in that lets you know as you're coding.

> If you write a type hint in your code and then run mypy on it, it will tell you whether the type is being violated.

Only if the type hints for all the libraries you use are correct (and in practice they're not).

Ah, thanks. I think I am going to give type hints another try.

Yeah, it’s always awkward to have to declare a variable or add an assert or a weird cast thing to make the type checker happy. Basically any case where type hints have to be evaluated at runtime for syntactic correctness.

The import pattern in the article is actually something I’ve never encountered because of how common the HAS_MODULE pattern is. I’ve never even thought to override the module binding.

This strikes me as extremely kludgey, inelegant and ugly. And this is coming from someone who rarely has qualms with pretty terrible hacks if they'll get the job done. I'd expect something like this to happen in BASH (pretty sure I've written approximately this in the past year), but it pains my heart to see this being done to a PyLang.

What a mess python can be, it is spectacular in this way. While it's possible with any tool, this deserves a special category award:

I hereby nominate thee probable annual 2022 winner of:

    HN Kludge of the Year

With PyTorch AI derived confidence of 0.99999

Care to share your model training code?

With confidence that high, it might be a runner up for kludge of the year.

I had to do a similar hack recently. The 'zoneinfo' library is available only on Python >=3.9. But the 'backports.zoneinfo' library is available for older Pythons. The recommended way of handling this causes mypy to spit out errors:

  try:
      import zoneinfo
  except:
      from backports import zoneinfo

The solution that I found to make mypy happy is:

  import sys
  if sys.version_info >= (3, 9):
      import zoneinfo
  else:
      from backports import zoneinfo

I have no idea why mypy is happy.

If I can get type hinting to work, it does detect certain errors in certain situations. But the whole thing takes too much work. The syntax is awkward and klunky. I'm not always sure that mypy actually catches typing errors, even with the --strict flag. And Python package management is already a shit show, but when I add mypy to the mix, it spews incomprehensible error messages that I have no idea how to fix.

I'm starting to look at Golang to replace my Python usage. If I'm going to do the work of adding typing info, I'd rather do it in a language that has static type checking from the very beginning. And being able to compile down to a single binary is appealing.

Pokemon exception handling is a bad idea in almost all instances.

The first import can fail for any number of reasons apart from the expected one and that reason will be swallowed and potentially replaced by a different, misleading, second order error message or worse - strange hard to explain behavior.

The second will behave (and more importantly, fail) in a much more predictable way.

It took me a few seconds to figure out what you were talking about with Pokemon. That's the first time I've heard that term! I love it.

(For anyone else scratching their head, it's Pokemon exception handling cause the except block catches any and all exceptions (gotta catch 'em all).)

Mypy has hard-coded support for `sys.version_info`.

And a command line argument to set its value.

If you're targeting multiple Python versions your CI should be running mypy multiple times.

> I'm starting to look at Golang to replace my Python usage.

I likewise switched to D and Rust and (excepting the difference in ecosystems richness compared to Python) am easily equally (or more) productive in statically typed languages.

Ah yes, nice hint that backports can use sys.version_info.

Or just use cast.

    from types import ModuleType
    from typing import cast

    try:
        import markdown
    except ImportError:
        markdown = cast(ModuleType, None)

What a terrible hack, but I also think try/catch with imports is a terrible idea. So in this case, the punishment for having a hacky cludge, is in itself having a hacky cludge.

Why is try/catch with imports a terrible idea? I mean if you need bignum, sure you can't run without the bignum import. But if you only want nicely typeset messages for humans, whats wrong with continuing even without it?

I think there are other ways to do it, like https://docs.python.org/3/library/importlib.html#checking-if... (though this only works in Python 3.4+ it seems, which might be a constraint and of course many codebases will predate this idiom)

The thing listed as “A Non-Functioning Alternative” seems to work correctly in both Pylance and mypy, if you add “# type: ignore” at the end of the import line.

Unfortunately that prevents Mypy from loading the type hints for 'markdown'

    from types import ModuleType

    markdown: ModuleType | None

    try:
        import markdown  # type: ignore [no-redef]
    except ImportError:
        markdown = None

    if markdown is not None:
        reveal_type(markdown.markdown)

Running Mypy:

    example.py:11: note: Revealed type is "Any"

Whereas when correctly imported:

    example.py:3: note: Revealed type is "def (text: builtins.str, *, extensions: Union[typing.Sequence[Union[builtins.str, markdown.extensions.Extension]], None] =, extension_configs: Union[typing.Mapping[builtins.str, typing.Mapping[builtins.str, Any]], None] =, output_format: Union[Literal['xhtml'], Literal['html'], None] =, tab_length: Union[builtins.int, None] =) -> builtins.str"

I'm not keen on this, as I don't like creating variables that only exist on one branch.

Would it work if you used importlib?

It will work but Mypy won't know to use the type hints for the markdown module.

Reply on news.ycombinator.com