Type hinting sucks

I'm usually a strict proponent of static typing, but I discovered that for the things I use Python for (mostly quick'n'dirty cross-platform scripting, not "real programs") it's indeed quite a nuisance mainly for non-trivial cases where the type hints looked entirely too complex and 'dominated' the code way too much resulting in (paradoxically) less readability. In the end I quickly abandonded my experiments and returned to no type hints.

I also find it kind of hilarious that type checking isn't built into the standard python interpreter, but you need a separate tool instead. It took me a couple of wasted hours until I discovered that my carefully type-hinted code was not actually type-checked, and after discovering that I was baffled that I can't just do something like "python --type-checked bla.py".

TL;DR: for many use cases, Python's duck typing is a feature, not a bug :)

slow_add is not simple. It reimplements the Python + operator, which is quite complex, specially if you look at it from a type system perspective. As noted, its implementation is very core to the language and it has several edge cases. That's not usually what you do when you create a function. Rather, you have very specific types that you support. It's only "simple" because adding things together is a basic operation.

Even if you write a function that encapsulates some funky arithmetic. You will probably be using more operations, to the point that anything but numbers will really make sense.

That said, yes, adding types to an API that wasn't implemented with a typing system in mind can be challenging. But I feel like you get a better API at the end of that exercise.

Groovy taught me that opt-in typing always sucks. Either a language is statically typed from it's inception or it's not. Type hinting is like putting up doors with locks around a house that has no walls to begin with.

Is anyone able to summarise how python got to this point? I haven't used it seriously since the tail end of the 2.7 woes but liked it a lot back then. Good native dictionary type and first class closures worked for me. Scattering of unit tests. The type annotation idea always seemed inconsistent with the language to me so I've ignored it.

In particular I'm wondering if this is a consequence of the rumours that python scales badly - someone changes some module far away, runs the massive test suite and it passes, checks in their code, and your code promptly blows up because the interaction wasn't tested.

There also seems to be a current enthusiasm for statically typing everything on hackernews which might be reflective of the wider industry, possibly making python acceptable collateral damage as it was on the wrong side.

That's my conjecture though, would love to hear from someone closer to the game how type hints became such a big deal in the python world.

I'm coming more and more to the conclusion that if you need typing for Python, you might as well go full throttle and use Java (maybe TS/Deno?), unless you're in a specialized ecosystem like data science and you have no choice.

Typehints are a great way to document code. If the type hints get too complex they can be cumbersome though. For example passing an int or None if the value doesn’ exist

I was expecting the article to get to the end and have him invent an massive type system that allows you to pass any two types as long as your can add them together. Which, of course, is what it did at the beginning.

By the fourth attempt the author seems to be half on the way to inventing typeclasses.

Opinion: people don't actually need rigorus typing system, what they need:

1. preven type/interface errors at runtime.

2. get auto-complete for unfamiliar objects or function signature.

I think (optional) static typing is important for "serious" programming, so I've been experimenting with Python's type hints after of my good experiences with TypeScript (which fundamentally works in the same way).

Oh boy, what a mess, especially if you have to keep compatibility with 3.7 (which has slightly different types for builtins).

The ecosystem is definitely improving and I do think the benefit will eventually be worth it, but this example illustrates (albeit in a somewhat exaggerated way) why I would advise users to move cautiously.

IDE completions are a good benefit for a start, it'll probably take a while until you can support strict checking (errors when types don't match).

I find this an interesting example to ponder, because I think it actually reveals a weakness in Python’s data model, more than in its type hinting—a weakness inherent in class-based programming that is also found to a considerable extent in just about every dynamic language I can think of, but which is made particularly obvious in Python’s operators.

The cause of most of the trouble here is that `lhs + rhs` isn’t simple sugar like a reasonable person might imagine if they don’t think through the implications long enough; but instead, turns into something like this terrific mess:

  def +(lhs, rhs):
      lhs_type = type(lhs)
      rhs_type = type(rhs)
      if do_radd_first := issubclass(rhs, lhs):
          if hasattr(rhs_type, '__radd__'):
              output = rhs_type.__radd__(rhs, lhs)
              if output != NotImplemented:
                  return output
      if hasattr(lhs_type, '__add__'):
          output = lhs_type.__add__(lhs, rhs)
          if output != NotImplemented:
              return output
      if not do_radd_first and hasattr(rhs_type, '__radd__'):
          output = rhs_type.__radd__(rhs, lhs)
          if output != NotImplemented:
              return output
      raise TypeError(f'unsupported operand type(s) for +: {lhs_type.__name__!r} and {rhs_type.__name__!r}')

(This is almost certainly imperfect in details, and may be imperfect in larger pieces; it’s a quick sketch based on memory from about nine years ago, plus a quick check of https://docs.python.org/3/reference/datamodel.html#object.__... where I had completely forgotten about the subclass thing. But I think it’s pretty close. Good luck figuring any of this out as a beginner, though, or even being confident of exactly what it does, because there’s no clear documentation anywhere on it, and the reference material misses details like the handling of NotImplemented in __radd__, so that I’m not in the slightest bit confident that my sketch is correct. If I still worked in Python, I’d probably turn this into a reference-style blog post, but I don’t.)

And why is this so? Because with class-based programming, the only place you can attach behaviour to an object or type is on that object or type. Which means that for operator overloading, it must goes on the first operand. But there are many legitimate cases where you can’t do that, and not all operators are commutative (e.g. a - b ≠ b - a in general), so you pretty much have to support the reflected operators, rhs.__radd__(lhs) instead of lhs.__add__(rhs), and then you get worried about subclassing problems, and it all just gets painfully complicated.

Is it any wonder, then, that a typing system would have trouble with it? Certainly Addable/RAddable are the wrong level of abstraction: you need a bound that covers both of them in one go, and I doubt you can do that with Protocol or similar, since you’re defining a pair of types, where one has this method, or the other has that method (and good luck handling NotImplemented scenarios). I imagine it needs to be built into the typing module as a new primitive.

—⁂—

By contrast, in Rust, you might possibly start with a concrete type (the first attempt), but you’d be more likely to just go straight to the fully-correct solution that this example is never able to reach in Python, of being generic with an Add trait bound <https://doc.rust-lang.org/std/ops/trait.Add.html>:

  fn slow_add<A: std::ops::Add<B>, B>(a: A, b: B) -> A::Output {
      std::thread::sleep(std::time::Duration::from_secs_f32(0.1));
      a + b
  }

Certainly Rust does have the advantage of having started with and built upon a type system, rather than retrofitting it. But note how this allows you to use different left-hand side, right-hand side and output types (A::Output is here short for <A as std::ops::Add<B>>::Output, meaning “the type produced by A + B”; Output is what’s called an associated type), and threads the types through fully properly. And note more importantly how this doesn’t run into the __add__/__radd__ problems, because the implementation isn’t attached to the single type A, but is rather defined, as it were, for the (A, B) tuple: when you compile it, it’s like you have a lookup table keyed by a (Lhs, Rhs) tuple. Depending on what sorts of additions the left hand side type defines, the right hand side type may be able to define additions of its own. (If you’re interested in how this is done, so that different libraries can’t define conflicting additions, look up trait implementation coherence.)

Rust thus demonstrates one solution to the problem this case exposes with using classes in this way: instead of putting data and behaviour together in classes, separate them.

(It’s not all sunshine and roses: some things do map to class structures very nicely, so that implementing them in Rust can be painful and take time to figure out a decent alternative, especially when interacting with existing systems; but in general, I find myself strongly appreciating languages with this sort of data/behaviour division.)

—⁂—

My favourite demonstration of the advantages of separating data and behaviour is actually iterators:

• In Rust, when you implement the Iterator trait on your type, you can call any iterator methods on it—built-in ones like .map() and .filter(), but also methods defined in other extension traits, e.g. https://docs.rs/itertools/latest/itertools/trait.Itertools.h.... This is why iterators are very popular in Rust: because they just work, with no trouble.

• By contrast, in Python map() and filter() have to be globals, leading to messy code reading order and the preferred alternative approach of list comprehensions/generator expressions (which work pretty well, but are more limited, really only covering map, filter and flat_map in their capabilities).

• In JavaScript, you get Array.prototype.{map, filter, …}, and those methods are defined as working on any iterator, not just an array, because otherwise it’d be just too painful—but unless you copy the methods you want (like NodeList has done with forEach, but not any other method!) you can’t just chain things automatically, and you can’t add new methods anywhere.

• Ruby has a… different approach to all this, but I can’t remember all that much about it and this comment is long enough already.

Type hinting does not suck. But missing support for method overloading does.

> Type hints are great! But I was playing Devil's advocate on a thread recently where I claimed actually type hinting can be legitimately annoying, especially to old school Python programmers.

> TL;DR Turning even the simplest function that relied on Duck Typing into a Type Hinted function that is useful can be painfully difficult.

Tbh, for smaller projects I feel this way. But when you're in a big workspace with lots of files and lots of variables, you forget after a while.

Isn't that a problem of types in general? I mean, you have the same problem in a strongly typed language.

This really triggered me for some reason! I hope it's a joke :)

(I am not sure exactly what triggered me but I think it's because at least to me it's very clear that in this very generic case, you just want to annotate with `Any`. Why go to such great lengths?)

the problem is not type hinting. the problem is people trying to use a single function for any type. as can be seen in the article, that is extremely difficult and error prone. so just dont do that:

    def slow_add(a: int, b: int) -> int:
    def slow_add_float(a: float, b: float) -> float:

OK its not as elegant or pretty, but its really clear what is going on. the problems described in the article crop up because people are trying to have the interpreter intuit what the programmer is wanting, when what the programmer is wanting might be ill defined or simply invalid.

* Declare `slow_add` to be used with instances implementing `__add__` protocol only.

* Declare that `concat` should be used to achieve a similar result with other types implementing `Foldable` protocol.

* Carry on.

It seems to me that the author isn't comfortable with static typing. The obvious solution (which might be impossible in MyPy?) is to have a two-parameter type class (protocol in MyPy?) of the lhs and rhs, and have the output be of an associated type. That should cover almost all cases.

It seems the problem stems from a poorly defined purpose of the API, which is to say, from this being a shitty example. Why would someone use slow_add instead of the plus operator anyway?