That's just how effective Python is done.
By using a static type checker on a dynamically typed language, you have admitted you don't know what are you doing right of the bat. This means the software engineers in the project are very bad and therefore the code quality overall will also be bad.
Tools like MyPy exist to make Python appear to be more like Java to help Java developers, who aren't willing to learn how to code in a different programming paradigm.
That code will always be far worse than Python code written using the Python development paradigms.
Conversely, I do think you're very bad and don't have much experience with large python codebases.
For example the suggestion to split and use microservices makes no sense. Microservices are even harder to refactor.
Python Microservices style when it comes to code quality, maintainability, etc... wipe the floor with Monorepo MyPy style projects. It's not even close.
So do our anecdotes cancel out?
If you have introduced static typing, you then have to start doing refactoring and verifying correctness. "heavily-typed mono repos which is a breeze to operate on" I highly doubt such a thing exists, more likely you are used to a certain level of bad code and don't understand it can be better.
Ohhh trust me, I know bad code. And I know good code. My unit of measurement is how fried I feel at the end of the day. Dynamic loosy-goosy python? Brain fried, constant debugging, little confidence in deploys. Static types, pydantic, pycharm, mypy, DI? I'm in the zone all day.
I don't think you are the arbiter of all code, so I'm not sure what grounds you have to tell me what my taste in code is.
Additionally, you would be better served by writing comments with less presumption in them. It makes the discourse more adversarial than it needs to be.
Typing is good, anyplace you can get the computer to check more of your work - the better. There are practical limits and trade-offs as always, but some typing is better than none.
Microservices are usually a terrible idea. I know they're popular, but I've only had bad experiences with them. I strongly recommend against microservices in most situations.
Sorry if your post was sarcastic and it went over my head.
So using type hints to annotate your flask api, etc. to generate docs is great. No problem.
The issue is that there have been a lot of Java developers switching to Python and they want to pretend that Python is a statically typed language (and use tools like MyPy) because it's more familar to them.
Python is not a statically typed language and treating it as one leads to terrible results. Especially in large projects.
The issue of course if that the Java developers have no point of reference on what a successful large Python project looks like. So they think they are doing great with MyPy when if you compare what they are outputting to a proper large Python project, it's pretty clear they are doing terribly.
When you use Python properly you write self contained microservices. Typing information exists but only on the external interfaces e.g. API. You also don't share business logic code between the self contained microservices because that massively decreases the maintainabily of the overall system. You show your Python code is correct by using Unit Testing and Mocking.
Basically, there is a way to do large Python projects and MyPy (and other static type checking tools) have no place in that story. It only exists to support people in making bad decisions on their codebase.
If other languages had comparable ML ecosystems then a different language may have been chosen. But at moment today there's no competitor anywhere close. One lazy metric, I would estimate 90%+ of research papers to ML conferences are in python.
If at some point you decide the way one of those sub-libraries works is wrong, could be done better, etc. then you can write a new sub-library that just provides the same public interface.
One of the reasons why Python is so successful is it's usage of dynamic typing. Obviously, if you lose static typing some else needs to take it's place, in Python's case, stronger encapulation.
That's...that's literally how type systems and interfaces work. You do know Python supports behavioral subtyping (Protocol), right?
It sounds like you certainly have had some bad experiences with poorly-written typed python. But that speaks more to those maintainers not knowing how to actually use types effectively, vs a shortcoming of static typing in python. Python's type system has plenty of shortcomings, but has plenty of escape hatches as well.
What you get with typing and interfaces is that all the code has an interface. Small sections of code have a large badly-defined interface.
The main reason people are using typing in Python is to support large Monorepos. Because everything is typed in them, all the code in the repository depends on all the other code in a spaghete dependency graph.
This results in the following issues:
* Small code changes lead to hour long unit test runs since most of the unit tests need to be rerun for every change.
* It's impossible to update the Python version since all the Python code needs to be updated at the same time.
* Long check-in times for changes running MyPy over 100k lines of code.
* Maintainability issues since code can't be updated without knock on effects all over the codebase.
Okay, so what are the benefits of using types in Python:
* On average MyPy type checking will catch 1 bug per developer per year, which wouldn't of being caught normally.
* It makes people who previously coded in statically typed languages feel more familar with the code.
Basically, if you look at the Pros vs Cons, you should ditch the typing checking. It's a net negative to the codebase.
You're writing unit tests wrong if they are taking that long. Unit tests should be quick.
> It's impossible to update the Python version since all the Python code needs to be updated at the same time.
I've done it, so, objectively not impossible.
> Long check-in times for changes running MyPy over 100k lines of code.
TFA addresses this. It can be greatly sped up with use of caching in CI.
> Maintainability issues since code can't be updated without knock on effects all over the codebase.
That's exactly why static types are better - automatic refactors.
> On average MyPy type checking will catch 1 bug per developer per year, which wouldn't of being caught normally.
I catch several per day, simply from IDE highlighting, in real time, and fix them immediately, which only works because of the type system. (maybe it's wrong to call these bugs at this point, it's more like proto-bugs which never even get checked in cause there is immediate feedback)
> It makes people who previously coded in statically typed languages feel more familar with the code.
I came from a fully-dynamic-everything python world, and types were a breath of fresh air.
You've clearly been burned by (a) bad codebase(s), but I think you are drawing all the wrong conclusions. All the benefits of narrow interfaces, easy refactors, high maintainability, can be had with static typed python. I will admit it's much more challenging to be really competent at it than it ought to be, but this has been improving rapidly.
Trying to bolt a static type system on a dynamic object oriented language literally violates any benefit you get from using a dynamic language in the first place. (that is to say, change things as they run).
I have no idea why people try to enforce the programming paradigm of Java on Python. if you want a huge, static program... write it in a static language, don't write it in Python.