Python's “disappointing” superpowers
lukeplant.me.uk
lukeplant.me.uk
IME this kind of "magic" very quickly loses its appeal when you have to debug it. The author's idea of libraries wrapping this stuff up so users don't have to care about it just doesn't pan out at all in my experience.
This is cool if it works. Good luck when it doesn't due to user error, and more luck if it doesn't due to a bug.
You need to develop a whole strategy to tell the user what went wrong and how, and why, basically from scratch; admitting that you didn't, in fact, execute the generator, and that they have to change it. because of that.
It's pretty awesome that it's possible. And it would be pretty amazing to write something like that. But... idk if it's a good idea from a developer stand point. I like being able to understand what's happening in a library with a single click.
var result = await (from c in Customer where c.orders.sum(o => o.price) > 1000 select c).ToListAsync();Fine! These things are based on metaprogramming; chances are you've used some of them.
* In nearly any language, nearly any ORM, for describing tables / models.
* In Rust, serde, and nearly everything you #[derive].
* In Java, Lombok, Hibernate, and most dependency-injection tools.
* In Java, most mocking / stubbing libraries.
* In Python, tons of stuff, from standard library (dataclasses, namedtuples) to various parts of Django, etc.
Judging by the sustained and widespread use of these things, they are not exactly complete failures.
Similarly, any heavy use of lambdas in java will make step debugging confusing. As will any annotations you may use. Which is largely why many people grow to hate annotations.
Which is all a fancy way of saying we learn the same lessons again and again. Used smartly, all of these are great tools. Defining what is "smartly" is a place of dragons.
On the contrary, static typing in Python is still extremely nice to have. When I was at the peak of my Python career, I had a bug where I changed the return type of a function to be a tuple, and somehow unit tests missed one of the worst possible invocations, leading to an awful failure that occurred after a payment was processed but before actually completing the task, causing it to be retried repeatedly. To be clear, like any failure of this nature, it is one caused by many different problems, and we employed many different solutions; we started paying attention to test coverage, we began using MyPy (it was still quite new; this was also in Python 2 so it needed type erasure compilation among other things) and we made our payment processing logic more robust to prevent processing the same payment twice even in the case of everything else (like retry logic) failing to stop it again. But the thing that sucks is, fairly simple type inference without any additional type annotations could've detected that sort of bug without a potential for false positives.
So I feel like it's silly the way that some dynamic language proponents feel like static typing systems on top of dynamic languages is all from outsiders. On the contrary, the relative weakness of MyPy is actually what ultimately made me quit using Python, and if I had to use it today, then yes, of course I would opt for the highest degree of type safety, as a primary concern above being "pythonic." I like pretty code, but I like correct code more.
Sometimes this does prevent "better" solutions from working, but in my opinion nearly 100% of the time that this is the case, it's because:
- The type system in question is not sufficiently advanced to express the types elegantly.
OR
- The approach is inherently not safe and probably not a good idea. Like patching the request object inside of middleware in Django, for example.
I think TypeScript proves that with a sufficiently advanced type system, even arguably bad ideas can be type safe. For example, the ability to express dotted object paths safely in modern TypeScript is pretty impressive, and lets you map out older JS that does this accurately, but in general that seems like an unnecessary trick that will just make your code slower at runtime for the slightest terseness improvement over alternatives, lacking a language with sufficiently advanced metaprogramming.
The thing is though, eventually this thought process comes true. If a given programming language community winds up bleeding members who are moving on due to the lack of better static type checking systems, then the people left will invariably be much more likely to be against static type systems.
> If a given programming language community winds up bleeding members who are moving on due to the lack of better static type checking systems, then the people left will invariably be much more likely to be against static type systems.
This reminds me of one of the things I hate most about the Python library ecosystem: libraries attempt to camouflage type errors resulting from changes like this. Something that should be an immediate runtime error turns into a subtle production bug. I discovered an error of this kind just this week while doing what I often have to do when refactoring someone else's code in Python: reading and distrusting every single damned line.
Which led to reading the source code of a third-party library, where I discovered something that I've seen often enough that I'm starting to think it's some kind of norm in Python libraries:
A function is documented to take a list of Foo. But if the value you pass is not a list, it doesn't throw an error. If the value is None, it uses []. If the value is a set or a generator or a tuple, it uses list(value). If the value is none of those things, it uses [value]. Now that it has a list, it looks at the values in the list, and for any value that isn't a Foo instance, it does its best to handle it. For example, if it's a str, it passes the value to the Foo constructor.
This might feel helpful for beginners and non-professionals just trying to get something to work on a single data set, testing everything by hand, but it amounts to passive-aggressive betrayal when you're trying to maintain a large codebase with multiple contributors. In the case I discovered this week, instead of an immediate runtime error prompting the programmer to make a trivial fix in their code, we had a feature subtly degraded in production for months. Very "helpful."
And yes generally agree that Paramus should be not be too accepting. It gets confusing fast.
Python has an Iterable type class for this.
https://docs.python.org/3/library/typing.html#nominal-vs-str...
https://docs.python.org/3/library/collections.abc.html#colle...
You could have introduced an UAT environment and done basic QA.
Type checking is not a replacement for QA testing. However, it is much cheaper and also finds bugs that might be incredibly hard to find through random chance or through structured smoke testing, because it does not depend on how common a given code path is. QA testing is even less likely to find issues than automated test suites, since at least you can qualify branch coverage with test suites (which still will not enumerate all of the possible code paths, and thus... Type checking is incredibly useful.)
So while type checking really is not a replacement for QA, QA is also not a replacement for type checking.
I ultimately quit Python for many reasons, but the desire to write more correct code was the big one. Not just me, but my entire team was burned out on Python. Investing so much into unit testing and increasing coverage, upgrading Python versions, and adding type checking as much as possible was still not yielding the benefits we'd hoped for despite how much effort it took. We eventually found much better success with the then-still-new Go programming language, which we used on most new projects going forward. There was still the occasional head scratcher in production, but deployments were as quiet as a church mouse. YMMV of course; there's plenty of success and failure stories with any programming language. I think we were all ready and willing to put a lot of investment into improving our robustness situation and just felt that we got more out of that effort with Go than Python for our programs. Python still has plenty of advantages too, so we didn't completely quit it; we wound up using Django and SQLalchemy here and there, but definitely most stuff moved to Go over time. (And personally, I stopped using Python for those things too, since moving on.)
It's a pity. I really like types but mypy is just not great. It's ok, buts that's not enough for python.
Obviously things like UTF-8 support, or maybe getting a clean lexical scope option, or other “fixes” are good. Performance improvements are good.
But there are lots of mainstream languages that have mature static type systems, strong metaprogramming facilities, higher performance, or all of the above.
I’d way rather see effort go into getting the package management thing figured out, or the static analysis stuff improved further than any big movement in the language proper.
Just my 2c.
Maybe I'm in a minority, but I use dynamic typing. I can haul data around in flexible formats such as JSON, asdf, pickle, and so forth, and manipulate them as dict structures. Python builds the structure automatically.
One of my first languages was Pascal, and so I understand the principle of letting your data structures and type checking do your work, and I still live that way when programming in C for embedded stuff. There may be better ways of doing what I'm doing, but there are also worse ways. From my own experience with getting software changed to suit my own development work, data structures and file formats are right up there in terms of why software is so inflexible.
It is certainly happening, and I'm not sure Python the language is all the better for it. I say that as a guy who explicitly mentioned this "Python Is Not Java" blog-post during my first job interview as a professional programmer, more than 15 years ago.
So maybe some stuff in the Dotnet ecosystem got started because Python programmers were forced to use C# and tried to make that more dynamic.
In Django, a model alreay contains all the meta data and you need almost no code to generate CRUD views and APIs from it. Can't do that in Dotnet. I think I could make something like that but why bother and take the pain?
When dotnet people say "my experience" then they mostly mean no experience beyond dotnet (in my experience). I hope you are basing your comparison on more than a glance or a weekend project trying out Django...
What type support is lacking in the ef pg provider?
> Metadata for the models is hard to come by (probably there is a way?).
I'm not sure what metadata means here exactly, you can configure models with either the fluent api or the attribute api
By metadata I indeed mean info on what a field means: Human readable names, descriptions, hints on validation or user input fields. You can argue that's not the ORM's job, but having some of that already in the ORM allows Django to have CRUD apis and views with very little code beyond the models.
I think you could make something like DRF or Django admin on top of EF by abusing assembly introspection. But so far nobody dared to do that.
And everything taking from lisp/other functional languages: lambda, map, filter etc. Which I really like
https://peps.python.org/pep-0612/
from typing import Awaitable, Callable, ParamSpec, TypeVar
P = ParamSpec("P")
R = TypeVar("R")
def add_logging(f: Callable[P, R]) -> Callable[P, Awaitable[R]]:
async def inner(*args: P.args, **kwargs: P.kwargs) -> R:
await log_to_database()
return f(*args, **kwargs)
return inner
@add_logging
def takes_int_str(x: int, y: str) -> int:
return x + 7
await takes_int_str(1, "A") # Accepted
await takes_int_str("B", 2) # Correctly rejected by the type checker
And Python modules/frameworks can be a lot more "typed" than you'd expect. Django models, forms etc offer more runtime validation than typesystems can easily achieve. Yes, at runtime. But between not having a build process and in general being more readable and concise, I feel like Python lets you run into the runtime error long before you get the same thing done "error free" in C#. Not that C# doesn't have runtime errors...
I don't understand this, what do you mean? TypeScript is such a success partly because everything is an object in Javascript. There's no need to distinguish between classes and "dicts", as you have to with Python. Another example: Python functions have args and kwargs, which make typing significantly more complex (as the article points out, Callable doesn't even support kwargs). Javascript doesn't, so typing functions is trivial.
Part of the point of static typechecking is documentation that can be verified by tooling. If you have no types in your codebase, and you decide to change something from a dict to a class, good luck! You're going to have to manually trace all the data flows through your entire application to figure out what's affected.
Everything is an object in Python too.
> There's no need to distinguish between classes and "dicts", as you have to with Python.
"dict" in Python (or at least Python 3, but Python 2 is EOL now so Python 3 is the only active Python there is) is a class. You can even subclass it, which if you want a customized dict for some reason in your particular application is often the best way to do it. Or you can subclass MutableMapping, which dict itself is a subclass of, and which is what is supposed to be used for isinstance checks now that Python has established abstract base classes for all of its standard "built-in type" interfaces.
Ah yes, true. JavaScript goes one step further though, and has no distinction between get-item accesses and attribute accesses. This makes TypeScript's data model much simpler to manage. Is it even possible to implement your own `MutableMapping` or `dataclasses` equivalent type in Python? I know the latter requires custom plugins.
Can you explain a bit more about what you mean here?
> Is it even possible to implement your own `MutableMapping` or `dataclasses` equivalent type in Python?
Of course. With dataclasses that's what Python itself does; that's a pure Python module:
https://github.com/python/cpython/blob/3.11/Lib/dataclasses....
For MutableMapping, Python currently implements it as a C class for speed, but you could implement the same functionality in pure Python. That's what earlier versions of the collections.abc module did.
> I know the latter requires custom plugins.
I don't know what you mean here. See the pure Python module that Python itself provides above.
> Of course. With dataclasses that's what Python itself does; that's a pure Python module:
Sorry, I meant that you can't create an equivalent class that typechecks the same way. For example, when you define a dataclass with fields, mypy and pyright are capable of inferring what args and kwargs the constructor supports. But that's because it was implemented as a special case. For example, see the mypy docs [1]. If you just alias `dataclasses`, you lose type support!
Pydantic is a library with similar semantics, and the only way to get typechecking support is to install the pydantic plugin for the typechecker you use. The language simply can't express it.
[1] https://mypy.readthedocs.io/en/stable/additional_features.ht...
Ah, ok. Technically I suppose one could unify these in Python; attribute accesses on objects are really dict lookups under the hood, and one could implement a list as a dict whose keys were restricted to be integers. But yes, Python chose not to go that route.
> I meant that you can't create an equivalent class that typechecks the same way.
Well, of course not. If you derive your own class direct from object, even if it's duck type compatible with some other class, it is not a subclass of that other class. That's just how Python's type system works.
> If you just alias `dataclasses`, you lose type support!
You mean you lose some functionality with third party type checking tools. Yes, that's true. (Although that particular limitation, at least, seems odd to me--see below.)
> The language simply can't express it.
No, the language does not have built-in tools that do the things you would like them to do with type checks and type annotations. And the third-party tools apparently don't cover these cases.
The "alias dataclass" case seems odd to me because a simple "is" check ("dataclass_alias is dataclass") should cover it--aliasing the object just means putting a reference to the same object in another namespace, and the "is" operator checks for object identity, not the name it has in any particular namespace.
The "dataclass_wrapper" case would be a good bit more complicated to check for, yes. Although type annotations ought to be able to help, since you can express "this function takes a class as an argument and returns a dataclass derived from that class" in Python type annotations.
No, I mean that Python cannot express this relationship with type annotations. That's why plugins are necessary, because the language doesn't support it. I think the same is true for `TypedDict`, which is why the community had to wait for official support before it could be used.
These fundamental limitations of the language prevent the community from effectively building new tools with similar behaviors. You have to either write your own plugin for the n different 3rd party type checking tools, or hope it gets pulled into an official Python version (at which point the n different type checking tools will implement support for you).
Anyways. This started off because of the claim that Python doesn't need typechecking as much because it has a simpler data model than JavaScript (at most, you can claim they are equivalent), but it's turned into gripes about how half-baked the type annotation syntax is. All of this to say: it could have been a contender! It could have been like TypeScript!!
What relationship? That two classes which have no ancestor classes in common happen to be duck type compatible? Yes, Python type annotations can't express that, because there is no type relationship to express.
The way to avoid this problem in Python is to make sure duck type compatible classes have an ancestor class in common. That was a main point of the collections.abc module when it was introduced, to provide actual classes that expressed the duck types that were built into Python (being a subclass of MutableMapping, for example, expresses the fact that a class is duck type compatible with the built-in dict class).
As for "dataclass", it's not a class, it's a function. But the dataclass module provides the "is_dataclass" function that you can call on any class to check whether it's a dataclass. As far as I know, this will detect both your "dataclass_alias" and "dataclass_wrapper" cases. So this function could be used by any tool that wants to check for dataclasses.
No, it's to say: "I think Python should have done things the way TypeScript does!" But Python is not TypeScript. It's a different language with a different design philosophy. Expecting to use it exactly the way you would use TypeScript is of course not going to work out well, just as expecting to use TypeScript exactly the way you would use Python is not going to work out well.
The typing module is pure Python, so the code for TypedDict in that module could have been implemented by any Python user. The official support is of course nicer, but is not required to extend the functionality of the typing module.
Not really. It's only complex when they're used with poor design decisions. There are plenty of languages that allow you to do things that aren't a good idea. Python is no exception. Type hinting just makes those shortcomings more readily apparent.
> Callable doesn't even support kwargs
PEP 544 and Protocols have existed for 5 years, and they support keyword only type hinting for callables.
Expecting to inherit type hints from an unrelated place, like the constructor of a superclass, is quite unjustified. The superclass is an implementation detail.
And while that is a simple pattern, it's not a good pattern, and quickly becomes unmaintainable in a program of any size. Using *kwargs because of Too-many-arguments and so you can violate the Liskov substitution principle isn't good programming. Just because it can be done doesn't mean we should start bastardizing typing to make it easier.
I don't think so. I just threw together a simple example that doesn't work:
from typing import TypedDict
class A(TypedDict):
name: str
age: int
def myfunc(**kwargs: A):
print(kwargs)
myfunc()
> And while that is a simple pattern, it's not a good pattern, and quickly becomes unmaintainable in a program of any size
It's not though, if you had a proper type system. For example, you could do this trivially in Typescript and it would automatically infer the types of the rest of the kwargs. That means that your type information is preserved, making it possible to keep using your subclass without losing information about the total set of arguments it supports. The advantage is that you can add/remove arguments to your base class without having to hunt down every subclass, and you're guaranteed to be correct (and it would error otherwise!).
We had an entire history of Python 2 without type hints. Why use them now?
Many of us in the python community know the benefits, and we’re simply changing our stack so python is as contained as possible, and dropping lower quality libraries for higher quality typed ones.
Many people in the Python community simply aren't writing those kinds of applications. They're writing applications where typing is not a benefit, it's a hindrance, so they don't use it, and Python makes that easy.
This is not to make the general claim "Typing is a hindrance in parsing applications", or anything close. It's saying "the current static type system(s) available in Python would have made this Python library much worse".
But often it can be helpful to know the shape of the input and the shape of the output. Eg are we working with a 2D,3D or ND array.
Before 3.11 it was not simple (possible?) To type this. But https://peps.python.org/pep-0646/ makes it possible. If somewhat clunky on the definition side.
And Python has a ton of newbies, college kids etc. Though this is also true of JavaScript.
As an industrial language/environment with best practices for very large codebases, it's really weak and I much prefer TypeScript despite the occasional tooling fatigue (has slowed down).
But a ton of money has been poured into JavaScript (v8) which really set that into motion.
Also, we've been using non-standard type hints for nearly all of python 2, they were just non-uniformly specified in docstrings and weren't easily referenced or introspected for tooling and IDE's to work with them.
Static typing, compile time checks just win in the long run. And with languages like kotlin you still have all the advantages, and you have nice tools like python has which execute at COMPILE time using DSL, which are type safe themselve..
Python has become mostly a toy language for me to write scripts in.
I've done significant refactors in untyped Python without much ado: as soon as your tests pass, code works too. I've done significant changes with tests passing on the first go.
https://learn.adacore.com/courses/intro-to-spark/chapters/01...
This is a general limitation of type systems, IIUC. If you want a type system to guarantee 100% program correctness, I think it can't handle an arbitrary Turing-complete program.
Disclaimer: I'm not sure I really understand this topic, so take this with a grain of salt.
At least now I know what is achievable despite its presence.
I doubt there is much relevance left, especially in light of Java's excellent functional integration into its type system and JVM.
Edit: ``` There's no reason Java couldn't simply add first-class functions and finally enter the grown-up, non-skewed world that allows people to use verbs as part of their thought processes ```
Ironically, this is exactly what happened. With exception that proper technological implementation took many years and might have one of the best implementations for anonymous functions out of any programming language (thank you, dynamic JVM).
By comparison using and modifying ORMs in languages like Python and Ruby feels like second nature.
~1-2k loc around the range of being easy to get up to speed on what's going on than equivalent 100k/10k LOC program with less 'typing' issues. (compiler/interpeter & keyboard too!)
Haven't run across a uni-py language yet (python & unicorn mash up)
[0] : https://en.wikipedia.org/wiki/Icon_(programming_language)
[1] : https://en.wikipedia.org/wiki/Unicon_(programming_language)
All joking aside, I do think Python is pretty handy.
Like this?
>>> class MyClass:
... attr = 0
...
>>> my_object = MyClass()
>>> MyClass.attr = 42
>>> my_object.attr
42
>>> MyClass.method = lambda self: self.attr
>>> my_object.method()
42
What is the difference with https://lukeplant.me.uk/blog/posts/pythons-disappointing-sup... ?
There are probably some subtleties and differences in how powerful both systems are (can't say more, I don't know how powerful python's metaprogramming is), but just know that Lisp has practically no limit in how you can extend the language.
Dynamic typing is excellent for experimentation though.
Python hits the sweet spot.
But I can definitely recommend Groovy as an excellent alternative having all the strengths of the Java ecosystem.
Scala and Kotlin are also quickly catching up.
While usually pitched as "only" a language for fast extension modules and/or bindings to existing C/C++ code, Cython is really a language in its own right. https://cython.org/ has some more details.
It works by generating C code that calls the regular CPython API, managing all the ref count and tuple jazz that's rather a hassle to write in C directly. Then you run a C compiler against this code and it literally just links against the CPython .so's/DLLs. So, it's not just calling module code the same way as CPython, but actually calling the core built-in to Python code the same way as well.
As you add more `cdef` type annotations to your cython code, the generated C code becomes closer & closer to hand-written C code (both in hazards and efficiency).
Of course, a caveat is that may be Cython also doesn't improve upon Python very much as a language, except for allowing gradual static typing.
There's a good Python -> Nim bridge: https://github.com/yglukhov/nimpy
https://github.com/vindarel/languages-that-compile-to-python
What management like very much is that using F# attracts better talent, but they no can believe.
It has great package/environment management, excellent (potentially close-to-C) performance, automatic type inference, and a good set of interactive tools that make for a rich REPL.
But any such FFI (from any language) is an additional point-of-failure, an extra gear that could break and has to be maintained. And on the con side of Julia, there's an initial compilation time (which is improving version to version, but still a factor to consider).
Also it should be mentioned that the advantage of Python is not only its ecosystem of libraries, but also the vast array of tutorials and learning resources, and of development tools. Those are areas where most other languages have to play catch-up with Python, especially a relative newcomer like Julia.
[1] https://github.com/cjdoris/PythonCall.jl [2] https://github.com/JuliaPy/PyCall.jl/
I tend to stick to vanilla python though, mainly because Hy is too much of an hassle for my use cases.
The thing is, once we go through all this static typing exercise in Python, we get no performance advantages, and the whole thing seems bolted on, with worse semantics than most modern typed languages. And yes, it can stifle undeniable advantages a dynamic language like Python can provide.
Python remains a fantastic prototyping and scripting language, but my feeling is that it now handles more than it should.
My experience has been that cython "just works" even with lots of external libraries etc, and the code ends up performing as well as any c++ code i could realistically write while being much more understandable.
an alternative is rust + pyo3 https://pyo3.rs
here's a web framework written with it, https://robyn.tech
the other poster child for pyo3 is polars, https://www.pola.rs
it's simply, amazing.
Amazing is not an overstatement.
If creating your own Python libraries in Rust is what you want, you would check out the first link I sent only, the one for pyo3.
But, if you want to learn Rust, you probably wouldn't start out with pyo3. You first install Rust with https://rustup.rs/ and then check out the official book, and the book rust by example, that you can find here https://www.rust-lang.org/learn - and maybe write some code on the Rust playground https://play.rust-lang.org/ - then, you use pyo3 to build Python libraries in Rust, and then use maturin https://www.maturin.rs/ to build and publish them to Pypi.
But if you still prefer to begin with Rust by writing Python libraries (it's a valid strategy if you are very comfortable with working with multiple stacks), the Maturin link has a tutorial that setups a program that is half written in python, half written in Rust, https://www.maturin.rs/tutorial.html (well the pyo3 link I sent also has one too. You should refer to the documentation of both, because you will use the two together)
After learning Rust and building some stuff with pyo3, the next step is looking for libraries that you could leverage to make Python programs ultra fast. Here https://github.com/rayon-rs/rayon is an obvious choice, see some examples from the Rust cookbook https://rust-lang-nursery.github.io/rust-cookbook/concurrenc... - when you create a parallel iterator, it will distribute the processing to many threads (by default, one per core). The rust cookbook, by the way, is a nice reference to see some of the most used crates (Rust libraries) in the Rust ecosystem.
If you are doing async stuff in Python, you probably need to look up https://github.com/awestlake87/pyo3-asyncio and https://tokio.rs/ - that's two libraries that Robyn uses (see Robyn dependencies here https://github.com/sansyrox/robyn/blob/main/Cargo.toml)
Anyway there are some posts about pyo3 on the web, like this blog post https://boring-guy.sh/posts/river-rust/ (note: it uses an outdated version of pyo3, and doesn't seem to use maturin which is a newer tool). This post was written by the developers of https://github.com/online-ml/river - another Python library written in Rust
I think it sucks for prototyping too if your prototype is constantly evolving. I have a WIP with about 100 files in it that I've been aggressively evolving over the past year, and I already want to rewrite everything in something statically typed, because any kind of change became pain in the ass. Too bad the project requires Python-first libraries.
2. This is a deep learning research project. Files are layers, pure functions, and model builders. Can't really turn it into microservices.
I've also found that Python developers tend to write highly dynamic code that is difficult to statically type, whereas JavaScript developers - even ones that don't use Typescript - have mostly realised that that is a bad idea (with some exceptions cough Vue).
I'm just so over having to guess what a function might accept or return. Life is too short to spend it constantly reverse engineering code because people can't be arsed to write proper documentation.
People who love dynamic typing seem to think their code is just “understandable” and it almost always isn’t
Using MyPy will on average double your bug count per software feature. You might not think it does that. But if you actually go and measure it, the bugs go through the roof. There are studies on this. It's for subtle reasons revolving around where bugs come from in code.
One of the biggest sources of ugliness is the "None". The standard way of declaring variables ahead of time is setting them to None. But then, the type hints just become these ugly unreadable Optional[ActualTypeofVar] everywhere.
I'm not sure this method is actually "standard". For many use cases there is an obvious "null" value of the desired type (for example, just set an int variable to 0), so there's no need to use None.
For use cases where you can't just initialize the variable to the "null" value of its type (because that value has some other meaning for your program), then the true type of the variable isn't just the type you're thinking (e.g., not just an int), because you need to be able to distinguish the variable having the "null" value from it not being set at all (the None case). And for that kind of use case, Python's type hints are correctly forcing you to declare that variable's type the way that reflects the actual situation.
* Python should have a way of declaring variables without setting them to None, so we can type hint them to their actual types. Pre-declaration is very important for a prototyping language, and needs to be done in a way that the variable is visible to dir().
* Come up with a cleaner syntax than the wordy Optional[]. Thankfully, they have recently moved on from the ugly Union[a,b] to the more readable a | b. Something more readable could be done for Optional.
a: int
b: str | None = None if foo:
x = a
else:
x = b
* code readability; all relevant variables are in one place
* Python is a prototyping language, and a very common usage pattern is doing dir(object) to determine the names of all variables that could be set to something.
class Foo:
my_int: int
x: list[str]
if foo:
x = []
x: list[str] | None = None
Have you used VSCode? or PyCharm? Their typehinting support is pretty great. At least as good as anything for typescript.
I think that is a bit of a false framing. To wit, any system of 100k LOC will be hard to get into. Even harder to make changes in. There is no getting around that. None. Even worse if you have many entities flying about these 100k LOC. Each change to an entity will be dangerous. Static or dynamic. Especially if they are persisted anywhere, as at that point it is all too easy to get static guarantees that aren't reflecting actual data.
You need all the help you can get maintaining large codebases and static typing is a huge help.
Static analysis is, of course, good. If that is static typing or otherwise. So is running the code making sure it does what you want when running.
What is not a huge help, is a byzantine type hierarchy that nobody understands. Which is all too familiar to me from any codebase I have worked on with folks that dived straight into the type system before they knew what they were doing in the system.
Many strong type proponents have moved to the categories view. If you know the category type of what you are doing, the idea is that that will prevent bugs. But... that hasn't been my experience. Often it just hides what is actually happening behind another layer of jargon. Jargon that is not native to the problem being solved.
I bet that if you did a survey of developers who have worked on projects of this scale, 90% would disagree with your opinion.
My experience, sadly, is the louder a dev on the team is about either dynamic or static typing, the more likely that dev's code is something nobody else in the team wants to work with.
Static is better than dynamic for large projects, in my experience.
My criticisms in this thread is that the static type brigade does not hinge on evidence. It is typically hollow claims and getting angry at dynamic languages for being obviously bad for lack of helping.
After 25 years of working on all kinds of codebases I’ll take even badly engineered statically typed code over dynamic any day of the week including Sunday.
Badly designed software with terrible class structure (FooModel, FooSchema, FooSpec, FooSchemaSpec, FooSchemaSingle, FooSpecContainer with inexplicable relations between them) is way worse than a well designed non-explicitly-typed software.
So in theory the language is statically typed but in practice case a large part of the project is 'integer typed' ..
If you have some experience in software engineering you should already know that.
Based on the "hard evidence" in this thread you are either ignorant or a time wasting pedant.
I’ve been writing in python for 18 years, and yet I can far more quickly get into a large typescript codebase than a python one of equivalent size.
Static typing is a tool I like. I don't think we have evidence that it is an obvious win. And I've bounced off of several typescript code bases hard. To the point that I currently hate typescript. Despite being impressed with some of its capabilities.
Which is again to say ymmv. Bad code is bad with or without static types.
To get a gut feeling of the system, i.e., to understand the 10000 foot view of a 100k LOC system, is orders of magnitude easier with statically typed languages than dynamic languages.
And that is mostly because those 100k LOC are done by probably dozens of developers and that leads to different styles, approaches, conventions, etc.
A statically typed language would remove a lot of those headaches, because it does force a semblance of structure, that can be easy to reason with.
A language like Python is awesome for a micro-services architecture though.
This is no different from any system. Want to know how your car works? Start with a simplified diagram and gradually add more details.
Cars and other equipment are an amusing case study. What is the strongly typed version of a car schematics? Why does it look so different from what we think the idealized software should look like?
WASM will probably kill TypeScript shortly.
The two other languages you mention, TypeScript and WASM, are more part of the same platform mutually benefiting from the progress than competing each other.
Source: I have spent years coding in C++/Java, then Python. I have migrated Java projects into Python
I have not had a chance to learn or use a language like Go. But production use of Python, including building large code bases is real. We do resort to numba, cython or using Python API to compiled code.
I'm now involved in converting large codebases from SAS to Python. I don't think I will have the luxury of choosing another language like Go, for a number of reasons.
Heck, just look at static typing in Python.
It means having a distinct type for any complex structure you pass around (think pre-normalization API params, post-normalization API params, slightly enriched post-processing data as separate type vs a dict of str to anything in Python), or anything you want to make compatible (think interfaces vs duck typing).
As soon as the types get complicated, the code bloat begins.
Only to a certain point, after which the network effects override everything else.
Also, "dynamic code generation" for "meta programming" ?
Some people like to know which type of meta programming is being used at a given point. This is something akin to "all dynamic code generation is meta programming, but not all meta programming is dynamic code generation".
One should be careful to not merge or eliminate words with subtle differences. They usually exist for a reason.