Reasons to avoid static type checking in Python
typing.readthedocs.io
typing.readthedocs.io
Python works great for cases where the scope of a particular function is tightly constrained around a tightly constrained set of inputs. In this scenario, it is relatively easy to test fitness for purpose, and that testing can replace static type checking as a practical matter. Many bugs sneak through in Python that C++ would find at compile-time, but that’s okay because the domain is so narrow that you can squeeze them out in testing with minimal overhead.
The minute you start trying to build a more generalized library of functionality in Python, the combinatorial bug factory starts to become a major drag. While you can add type checking, it is clumsy, slow, and less thorough in practice than you would expect with C++. Python is not designed to provide the degree of static type checking something like C++ can. Many types of code benefit from thorough static type checking, and for those use cases something like Python has a high cost.
This is fine. I understand the problems for which both languages are best suited. Right tool for the job and all that. Static type checking is generally beneficial in software but it isn’t always worth paying the tariff if the problem domain is narrow enough. It is quite possible to overfit static type checking to simple cases that don’t justify the effort.
So if you ask me, who has been programming Python since before new-style classes, if types make sense for Python, the answer is a resounding ‘yes’: just steal as much typescript typing as possible.
The JS/TS community has enthusiastically adopted static typing, to the point where (a) the type system in TypeScript is reasonably clean and intuitive for everyday work, (b) almost every major library uses static types and specifies them in its documentation, (c) almost all major tools and development environments pick them up and work well with them, and probably as a result of those things, (d) almost everyone now uses static types extensively, at least for new development.
The Python community seems far more hesitant. I don’t know why. Maybe it’s because we’d just had such a long transition from Python 2 to Python 3 and introducing static typing feels like another big change? Maybe it’s because Python’s static type system simply hasn’t been as clean and tidy as TypeScript’s, particularly before the recent generics changes in 3.12? Whatever the reason, in practice none of (a)–(d) above reliably holds in the Python world. Personally I do still prefer to use static types in new Python code, but it’s frustrating and often needlessly time-consuming when popular libraries don’t.
And I hope one day iter[] and callable[] will be made available, cause importing typing sucks.
Just think of a function which generates a random number for a lottery app. Do you want to call the function 'generateRandomNumber' or 'generateLotteryNumber'? There is no reason to constrain your function to the 'lottery' use case when it could also serve an infinite number of other use cases.
The function which is used as a dependency shouldn't concern itself with the specific business case towards which it will be applied.
As you move towards the leaves of the dependency hierarchy, the abstraction becomes more about technology and less about business domain.
Maybe the dependent logic already has a similar type to represent URLs but maybe it's called URLInfo... The property names might be slightly different. Maybe URLInfo has a Host and Port property starting with capital letters but URLDefinition properties are all lower case.
IMO, this is a huge mistake. URLs should simply be treated as plain strings that way you don't need to worry about unnecessary interface mismatches. Inventing complex interfaces instead of just representing concepts as primitives adds unnecessary friction.
Parent components give their children the necessary info to do their jobs but the implementation details of how they do it is fully encapsulated within the children components.
When components start passing other components to each other, they're no longer merely communicating with each other, but they are attempting to micromanage each other's functionality. It's a failure of the separation of concerns principle. There are always alternative approaches; for example, the child can emit/trigger an event to let the parent component know that a low-level condition occurred, but let the parent decide how to handle it because the parent is responsible for that higher level of abstraction.
This is not a new concept I invented, it's essentially what Alan Kay (the inventor of OOP) said decades ago. He said that OOP is about messaging.
There can be exceptions to the rule, but the idea is to keep those exceptions to a minimum.
Regardless - this is very much a strawman. I could (try to) pick apart the example, but I'm not going to. Sometimes a complex interface is required, sometimes a simple one.
Therefore, I think "... the authors have no desire to ever make type hints mandatory, even by convention." is useless. I'm sure the authors are honest about their desires, but frankly either everything gets type checked or practically nothing does. It is unlikely that there is a social equilibrium where some things get checked and some do not. Not only that, but clever libraries that aren't written with type checking in mind are often a devil to hint retroactively.
Which, in turn, leads to you ignoring type errors entirely, since you don't know which one are spurious and which ones aren't.
In fact, type inference is a thing.
Most of the languages with Hindley Millner type system (such as oCaml, F#, Haskell, PureScript) do feel dynamic (e.g., developer does not need to hint the compiler) yet they are purely static typed.
What is the argument against having Python checking the types, without asking the developers to change their code?
- I avoid complex function/method interfaces. My coding philosophy revolves around relying on simple primitives as arguments and return values and keep complex state fully encapsulated.
- I use a TDD e2e approach which gives me excellent code coverage and catches 'type mismatches' early as such issues cause functionality problems that are caught in my tests. Type checking is redundant.
- My software is made up of small modules. The lower level modules are generic enough that it doesn't make sense to restrict to specific types or assume a specific business domain.
At a psychological level, I find that type checking tends to encourage developers to design complex function/method interfaces which lead to worse, harder to maintain software and encourages leaky abstractions and passing around 'live instances'. It encourages developers to add unnecessary constraints which make low level modules less versatile and cover fewer use cases than they could otherwise have.
Interface names of low level modules often end up unnecessarily referencing the specific business domain which should be a higher level concern.
You end up with thousands of lines of code that do very little except for being the physical incarnation of "the process" god.
That stuff is so 2000.
Type systems give you robustness and whole program consistency proofs without a single line of extra code. No manually writing tests to ensure type safety and you get help writing semantically meaningfull functions, instead of whole object graphs that need to be called in a complex dance just to avoid having 4 function arguments.
I've built a decentralized cryptocurrency exchange that can work with most blockchains in just 5000 lines of code. Has been running for 4 years with zero bugs.
I've written a cryptocurrency from scratch with only about 4000 lines of code and have not had any bug reported after 3 years of continuous operation. On the other hand, projects like Bitcoin and Ethereum are hundreds of thousands of lines.
I've also written a no-code platform in less than 4k lines of code which supports complex views with real time updates, pagination, indexing, access control, multi-tenancy and has been bug-free since it launched. I built the whole thing in about 3 months part time.
All of these built with plain JavaScript/Node.js. I can support all of these projects in parallel because I rarely need to update them.
Not really a good project to measure ossification on then.
Getting rid of dumb tests is often where the real line savings is. No need to test your specialized internal function that's just wrapping filtering an array. Like, yes, array filter does in fact filter according the predicate you give it.
I'm a big fan of type checking in general, and Typescript is a great example of a type checker that allows you to statically define all the dynamic idioms that you'd normally use. But for various reasons, that just doesn't seem to have happened in the Python ecosystem. Instead, you can either write normal dynamic Python without types, or you can restrict yourself to a vaguely Java-like subset of Python with hard-to-use generics and confusing rules about forward references.
I wonder if it would have been better to have taken the Typescript approach and have Mypy and all its annotations developed as a separate language, rather than as part of the Python language, with the expectation that they could eventually be merged at some point. As it is, it feels like both sides have been very constrained: Mypy has struggled to innovate on expressive type-level syntax because it only operates on standard Python code, and Python has struggled with balancing the various goals that different groups have for youry annotations.
The result is a kind of mess where it's still painful to type basic things like generics (yes, that's changing, but not if you need to support older Python versions), but you also have weird syntax that requires type annotations, even if they're meaningless (like with dataclasses).
I don’t understand what you’re getting at, but clearly you’re expressing some pain here: what are some examples?
> Your codebase is old, large and has been working fine without static type checking for years. While Python’s type system is designed to allow gradual adoption of static type checking, the total cost of adding type annotations to a large extant codebase can be prohibitive.
That's a convoluted and quite dishonest way of warning that you're trapped if you don't enforce types from the start; but it does have the merit of being on the list.
I’ve always been using mypy, but heard good things about pyright. It’s what VSCode uses by default as well, unless you jump through hoops. So I have pyright in the IDE and mypy everywhere else, yet they don’t agree with one another in various cases. Exhaustiveness checking and type inference in structural pattern matching, and async TaskGroups are some examples. One of my code bases has mixed “type: ignore” and “pyright: ignore” (forget the exact syntax) for that reason. Pretty bizarre and not something you’d see in any other language!
I wonder why mypy isn’t the blessed standard and pyright was able to catch up on (and in parts overtake) it. Mypy had a large head start and enjoys tons of development effort. Good job to Microsoft I suppose?
https://mypy.readthedocs.io/en/stable/mypy_daemon.html
(But I also await ruff type checking)
I've dealt with my share of code with types that were almost correct but not really and it's been quite frustrating.
For interested I've written an article about it: https://medium.com/@sgorawski/python-types-have-an-expectati...
Python is a motorbike. It is small, convenient, nippy, fuel-efficient. It is also terribly unsafe, holds no luggage and no protection from the weather.
Statically-typed languages are cars. Stable, safe, predictable, require a lot of infrequent maintenance, weather-proof. Not convenient in heavy traffic, but hey, you put on the heater seats and the stereo and you don't mind queuing.
Now, I find the effort to add typing to Python like trying to make the motorbike safe. You double the wheels to have a full 4-wheel vehicle, add a roof and windshield... And you end up with something that has th upsides of neither a car nor a motorbike.
Where people find it convenient, sure. But the evangelism of "types everywhere" seems to not get it.
Anything, including gradual typing, can be done well or not well.
Python is strongly typed, but it does not mandate static type analysis -- unlike most other typed languages, where it is mandatory at compilation time. But if you choose to use it, there are tools which will run it for you at any point in time (mypy being the reference implementation). You don't even have to write any special annotations for that: the checker will try to infer the type when possible, or use Any when it isn't.
The crucial thing to understand is that Python is still dynamic at runtime -- there are no checks that an argument is of the correct type defined for the corresponding parameter. So your program can fail the static analysis but correctly run in practice, unlike most other statically checked languages.
Another effect of the above, which is often overlooked, is that many developers tie themselves into knots to get the type annotations right on unit tests. This is completely missing the point, as unit tests and static typing are at completely cross purposes: while static analysis evaluates, as the name implies, the code in its static state, the unit tests only make sense when they are executed, i.e. at runtime.
So the practical rule of thumb, learned from practice, is:
1. Regularly run the static analysis on your application (and library) code, adding any necessary annotations, to weed out various problems that might arise from mismatched types.
2. Write thorough unit tests -- ideally with plenty of property tests with tools like hypothesis [0] -- and run them regularly.
^^ this excerpt from the post made me spit my wine out ... this is what https://raku.org is for!
Surely, according to the zen of python https://peps.python.org/pep-0020/, the pythonic way is:
There should be one-- and preferably only one --obvious way to do it.
At least raku is architected on consistent class types from the get go so its gradual typing is not just an annotation / type hint bolt on.
Heck yeah! Why would I want to find problems ahead of time when the whole thing could blow up at runtime with cryptic errors. Or better yet, no errors because I’m such a ducking elite programmer that I never make mistakes. Read the code. The code don’t lie!
This is a lame, low effort blog post. Jesus Christ.