It fundamentally can't be improved in a significant enough way and still be Python. The more realistic options are to be okay with Python's type system (I personally am) or to look at different languages if you really want something more robust.
Given how opinionated the Python maintainers can be, it baffles me that they accepted to get these optional, noisy, half backed type hints into the core language.
In my experience given that they're optional and you'll almost never get 100% of your code and its dependencies with correct and up to date signatures it's just a nuisance. Maybe in the right projects if all the devs are very thorough with them it can be helpful, but that's really not my experience. If at least it triggered an assertion at runtime when the type doesn't match it would be massively more useful. And even then, if you're so thorough with your typing, why not just use a proper statically typed language?
I really don't get why it's even there to be honest.
Not quite. You can reflect on and interact with them at run-time, too. This does make it possible to implement run-time checking as a library solution.
I can't speak to why run-time checking wasn't built into the language as a default behavior, and the PEP doesn't explain why (though perhaps the answer is in the mailing list), but one possibility is that it would be wasteful. Most functions that are only prepared to accept certain types already have some form of run-time type check baked in, either explicitly or incidentally, so adding a second check would likely introduce overhead without producing a whole lot more type safety.
tldr: you other get either huge overhead or need to type annotate everything like it's Java, and it'll still be pretty limited/not super useful
---
there's a bunch of libraries that do this, and they're all fundamentally limited [though i guess some of these may just be failures of imagination on my part]:
- hard to check generics: if it's a builtin, you have to traverse the whole structure and check every element. and for user-defined collections, the programmer has to to manually instantiate every generic like:
CoolSequence[int]([1,2,3])
- if your function takes a callback, there's no way to typecheck that callback at runtime (without calling it, at which point it's not terribly useful. which extends to all functions i guess)
- not much you can do with iterators/generators. can't check if something really is an `Iterator[int]` without exhausting it, and e.g. type-erroring after processing 20 elements because it yielded not-an-int isn't very useful
so in the end, even if you wrap Every Damn Thing in a type assertion, you still get the errors very late, not much sooner than your usual "AttributeError: object None has no property named 'foobar'". kind of a worst-of-both-worlds situation
Still using Java 8 are we?
If you do the legwork, you can get mypy[0] type-checking done on your codebase in a way that is similar to how TypeScript does type-checking. There are stub files that are provided by project maintainers that are then used by mypy to infer things like e.g the return type of a function.
Type hints are also inline in the code, and not technically comments. They can be retrieved from class attributes using facilities from the standard library [1] and can facilitate other tooling that is specific for your project or that are more general.
> And even then, if you're so thorough with your typing, why not just use a proper statically typed language?
That would remove a lot of the benefit of choosing Python. Python is dynamically typed. Type hints make it possible to do type checking statically, but with a lot of extra leg work (as I described above). Making Python itself statically typed is not something that would interest 99% of the Python community IMO.
[1] https://docs.python.org/3/library/typing.html#typing.get_typ...
Writing type hinted python in Pycharm or Ipython feels like racing in a sports car. Without it, feels like stop and go traffic with the occasional car accident.
I can see how if you are using a WYSIWYG editor, type hints feel like more text for little gain. But even then, it really helps document intent. In fact switching to IDEs is partly what made me go from someone who hated static typing to loving it.
This is a beautiful analogy!
As for the second part, though, I would say that even if your IDE does not provide automatic type checking, type hints will still add some benefit because:
- without type hints you have to do all type checking in your head (across all stack frames).
- whereas with type hints you only need to do type checking down to the first level of the stack.
To illustrate what I mean, consider the following code:
def bar(param1, param2):
... # Some complicated code from which it is not immediately clear what values are allowed for param1 and param2, nor what the return type of the function is.
def foo(param):
return bar(param, param2="some default value")
foo("something something")
Guido joined the Mypy team while he was Python’s BDFL.
Also, there’s nothing half baked about either Python’s type hints or Python’s type system, or it's major type checkers. It's not Haskell, sure, but it's an expressive to system, the typecheckers are reasonably smart, and the annotations are readable and sensible if somewhat verbose; there were some infelicities regarding alternate names for core types in annotations, but that's been improved recently.
> In my experience given that they're optional and you'll almost never get 100% of your code and its dependencies with correct and up to date signatures it's just a nuisance
In my experience they start to provide value in preventing bugs and easing development because of tooling support way below 100% coverage.
> If at least it triggered an assertion at runtime when the type doesn't match it would be massively more useful.
Python’s type annotations are annotations, and are used by some libraries for runtime (validation, serialization/deserialization, etc ) as well as static checking purposes (e.g., pydantic.)
> And even then, if you're so thorough with your typing, why not just use a proper statically typed language?
All a “proper statically typed” language is is a language with a static type checker run ahead of time, which Python is of you choose it to be. There's a lot of code in the ecosystem that is more broadly types than it needs to be, because no annotated code which checkers can't infer anything better for use Any, but that's evolving over time as it is more common for popular libraries to be typed, or at least have typings available.
This was always kinda false (type hints can be composed unlike eg docstring param @type). But with the introduction of dataclass, this is empirically false. You can readily hook into the type hints, reflect on them, use it to marshal data, etc. Using __post_init__, you can leverage any kind of runtime checks you want to enforce invariants.
Also type hints greatly help transpilation.
You don't need 100% to reap the benefits. In fact I will annotate small bits of un-typed code and it makes it way easier to reason about.
The chorus of people asking for increased typesafety had become too loud to ignore. Type hints gave them enough support to cover 95% of their needs, which were/are largely organizational rather than technical: developers must feel sure they are using the right classes in their code, pulled from the right places in their project, and don’t have to look up docs at every step (because the IDE will autocomplete stuff). What happens later, the low-level implementation, it doesn’t really matter; what matters is the information about types is surfaced somewhere, so that IDEs and tooling can use it to document projects and help developers.
That’s what type hints do, and yeah, they are basically glorified docs, but integrating docs into syntax is one of Python’s many traditional strengths (see docstrings, doctests, etc). That’s also why they are optional, thank goodness, so people who don’t have big-org / big-project needs can still be productive.
There are some interesting further add ons that seem very python, allowing you to go further. For example, with a pandas dataframe you can just say your type is a dataframe which isn't so useful, but it's possible to hack your own types onto it in the vein of https://github.com/CedricFR/dataenforce, or use things like https://smarie.github.io/python-vtypes/ to get smarter typing on things the authors didn't type. I expect that trend will continue.
What fascinates me about python's types is actually the very fact that they are bolted on. You have a language that lets you do crazy things and a type system trying to catch up and make it convenient to verify those crazy things. It's a nice complement to the usual developments of verifying all of the things and slowly extending the set of things you can do.
The difference between these two cases is that a function that returns [a] will always return [a] and if you convert it to `NonEmptyList n a` you will be nudged towards handling the case where it didn't actually return n elements, whereas if you annotate someone else's list as returning List[str] then it may well silently not do some of the time, and you won't get an error until a long way away.
And by the List[str] bit, I mean you can annotate it as doing whatever it in fact does. You aren’t protected if you get that wrong, of course, and that’s where things like stubs come in to help.
Yes - but as I said, you can be confident that the list type is accurate, and if you want to convert that list to a specific-length list type then you're nudged towards doing it in a way where you handle the case where the list isn't actually the length you specified.
> And by the List[str] bit, I mean you can annotate it as doing whatever it in fact does. You aren’t protected if you get that wrong, of course
Not "of course"; the whole point of a type system is that it's automatically checked and therefore you can have reasonable confidence in the types you're given.
Why not?
Static enforcement of invariants that doesn't change the space of what is possible in the language when you aren't asserting conflicting invariants doesn't make Python any less Python.