I feel like the dynamic typing crowd are slowly coming around to the fact that having a typesystem is just better.
While we’re on the topic, a better typesystem is better than a worse typesystem. Thanks for coming to my TED talk.
While we’re on the topic, a better typesystem is better than a worse typesystem. Thanks for coming to my TED talk.
That said, I’ve found Python’s approach lets me still build things through experimentation then fixing the types when I’m a bit more confident that I’ve built the right thing.
As a comparison, I found Go really clunky when I tried to learn it because it wouldn’t compile if the code wasn’t totally correct. It makes it hard to find the right solution through experimentation.
man I wish my coworkers did this step
Python is probably an all-round better language than JavaScript. But typescript absolutely embarrasses Python’s “gradual” type system with just how good it is in comparison.
And it does matter when 3rd party libraries are involved. Types are a wonderful gift to anyone who imports & needs to figure out your library.
That's a fair point.
Currently, I’m using the third-party Beartype decorators to provide this feature, but I would prefer something built-in and ideally something with more readable error messages for users when type errors occur, which might be easier to provide if it was built into Python.
(I use Pyright too, but I find the runtime type checking to be more robust since it understands also the types returned by highly dynamic untyped code.)
1. You need to understand how to express concepts in a form the type system can understand. Each type system will have patterns it understands, and patterns it does not.
2. Go isn't a great example. It has a Java-circa-2000 type system. Modern type systems are much more expressive.
I picked Go as an example of a language that gets a lot of praise specifically for having strict compiler rules.
I've also tried Scala and found it clunky but I do suspect most of its problems came from the insane operator overloading and opaque build system rather than the type system. I understand it's gotten better since I looked 8 years ago but I've not had a reason to check it out again.
Python with types is a double edged sword. If you can enforce other collaborators and SDKs use strict typing, it's a very pleasant experience to write, coupled with Pydantic for defining POJO-like structs. For CLI tools Click + Poetry build system.
Java17+ type system is pretty good. Way better than Go in terms of dev experience. you may need a POJO generation tool (Lombok, Autovalue) and preferably validation libraries. I'd still write java over Go if efficiency and single binary is not a concern.
Once you're using types, I would recommend Typer instead. Built on-top of Click but configured via type hints:
To me this suggests the tooling was insufficient. One of the things I value about type systems is being in a tight feedback loop with the compiler. Strong typing helps me run more experiments and to run them from within my editor. But without a good LSP and such, it would be miserable.
Perhaps but I find that type systems usually introduce friction to getting things working that, for some types of projects, can be worthwhile and, for others, not worth the overhead.
I'm not dogmatic, type systems aren't the end all. But I think a lot of the trouble people have with them comes from using the approaches they've developed in untyped systems and saying, "this is the same, I just get more warnings and errors." But the opportunity is in developing new approaches and habits that leverage the type system.
For me the "aha" moment was watching Jon Gjenset's YouTube videos, and seeing him intentionally write code with type errors in order to figure out what the type of something was. This helped me make the switch from thinking of errors as drudgery to be avoided to errors as feedback I may want to elicit.
Now I'm able to answer lots of questions within my editor that I'd previously need to use a REPL or consult the docs to answer. It's like I'm able to query my codebase, but the query language is just the language I'm already working in. The magic of strong typing is good tooling.
We shouldn’t pretend that it’s free. The fact is, sometimes running some code that I know works despite what the type system is telling me is very useful. Comparing to, say, Go, where I need to deal with everything (even if it’s incidental to the actual problem) adds friction. Sometimes I don’t want to deal with edge cases because this code will never leave my machine!
I can recall times when I've neglected a special case to write a one-time script faster, but they were always external to the type system. Eg, "this regex is pretty sketchy, but I know it will work with this particular set of logs." So I'm confused on this point, am I missing something?
All programs need to be type checked, even one-off scripts written in dynamic languages, to ensure that they function. The difference is whether the process is manual or automated. The computer is going to do it better and faster than I can, and that will free up mental bandwidth that I can apply to the problem at hand.
In a statically typed language you’ll get an error in the compiler. In Python it’ll run. Reading a file can be similar. These small frictions add up when I’m just trying to make a simple script work by experimentation.
At the risk of repeating myself: I think typing is a powerful tool but to pretend it has no trade-offs is 100% disingenuous or naive. If it didn’t, dynamic or untyped languages simply would never exist!
As far as the origins of dynamic typing goes, the amount of compute we can dedicate to tooling today, and the development of language servers in particular, radically changes the tradeoffs we can make. Strongly typed languages are better positioned to leverage tooling, and in my view this has completely upended the landscape of language ergonomics. Strong typing might not be free, but it does buy you a lot.
Does that view really make me naive or disingenuous? Perhaps we've just had different experiences. To be frank, when you say something that I solve in a blink of the eye is a serious friction, I have a similar reaction.
So you're telling me that the following:
data = requests.get(some_link).json()['the_field_i_care_about']
do_something_with(data)
is just as much effort as defining the types and dealing with the error case (depending on the language). You're either vastly more intelligent—and a faster typist—than pretty much every human being I've ever interacted with or, more likely, you're lying to me and/or yourself.
I'm not any more intelligent or a faster a typist than the next guy. Which is why I place such an emphasis on tooling (like code completion).
If you haven't given VSCode or another editor with LSP capabilities a try, you won't regret it.
1. It doesn't remove all friction—it can't magic type definitions out of thin air.
2. The fact that tooling exists means it's there to remove friction. Ergo, that friction exists.
3. Following 2: this tooling isn't appropriate in all environments. There's situations where opening vim and sketching out an idea is the fastest way.
Personally I also find python typing useful in "smaller" programs and I use them pretty much everywhere.
For example I use it in all my automation scripts which are usually not much longer than a few pages or a few modules. Adding type hints is a very minor task, really comes straight out of brain just like the code i type.
Type hints have given me wonderful completion and documentation which has helped me to write more correct and bug free code. Often little things pop up that would only have been caught at runtime. Now I see those things as soon as I type them.
Python + Type Hints == Awesome
Go won't even compile if a variable is unused.
Makes it really hard to comment-out lines when debugging.
Very honestly, optional type systems tend to be the worst of all worlds. Because people who don't care, don't need to use it, you don't get safety. But people who enjoy ceremony can inflict verbosity on others. While missing the most important reason to do it.
I can prototype, script, move fast. Then I can solidify, stabilize, collaborate.
* To get top performance, of course you'll need compiler, types, static dispatch.
It’s amazing how quickly a few interfaces / typedefs repay the time you spent typing them in.
Why not make it easier for yourself and be able to turn that into
> The compiler will tell me when this object won't
Very much. The “problem” people try to solve with things like dependency injection isn’t even a problem in python. 99.99% of the time you can just import you dependencies everywhere they are needed.
So many times now I’ve had to unteach bulky OOP patterns for people coming from strictly typed languages to python believe are “good practices” or are “needed for reliable code” and you go from 20 different interdependent classes down to 3 functions that just directly do what they are supposed to do. And you always end up with someone being disappointed that all of these complex patterns they learned to solve problems that don’t exist in python aren’t needed, rather than someone just happy that you can proceed directly to a value adding solution and skip the crud and design pattern spam.
And then they start arguing crazy stuff like the 200 extra lines unneeded code makes the solution “more readable” or “more reliable”, when in reality it’s just a desire to use the solutions they are used to even when the problems don’t exist.
> The “problem” people try to
> solve with things like
> dependency injection isn’t
> even a problem in python.
> 99.99% of the time you can
> just import you dependencies
> everywhere they are needed.
Consider a function with `import datetime`, that gets the current time and does something with it. You want to make sure that this function still does the right thing in the extreme case of the current time being 23:59:59. How do you write this test without DI?
from datetime import datetime, timedelta
def f():
now = datetime.now()
future = now + timedelta(seconds=1)
return now.time() < future.time()
from unittest.mock import patch
with patch('__main__.datetime', spec=datetime, side_effect=datetime) as mockdt:
mockdt.now.return_value = datetime(2024, 10, 3, 23, 59, 59)
assert f()
from datetime import datetime, timedelta
def f(now = datetime.now):
timestamp = now()
future = timestamp + timedelta(seconds=1)
return timestamp.time() < future.time()
def mockdt():
return datetime(2024, 10, 3, 23, 59, 59)
assert f(mockdt)
Also, this approach scales poorly with the number of dependencies, and that’s a good thing. If `f` were to also depend on a `db_connection` in addition to `datetime.now`, then this pattern automatically alerts the API designer that it might be time to redesign `f`.
Expecting type annotations to be more generally useful is mostly projected, baseless declaration anxiety: tools and methods that are useful in other languages are expected to be relevant in Python, certainly comforting for the type-addicted programmer but not necessarily useful. Declaring types is perceived as normal, as a necessary burden and dynamic typing is perceived as missing important structural elements.
Consider the error pattern of accessing a value as if it had a different type: in C or C++ consequences are dire (possibly undetected and cascading memory corruption), likelihood is very high due to specific language features (pointers and references, raw arrays, casts, weak typing in general) and detailed type declarations are a useful mitigation because they turn run time catastrophes into actionable compile time reports, while in Python, thanks to robust duck typing without dangerous complications, consequences are mild (reasonable error messages or wrong results, before corrupting memory), likelihood is low (specific instances of unexpected and malformed data or gross API misunderstandings) and type declarations do nothing.
I'll keep my dynamic typing, thanks!
There's a world of difference between the experience you get with untyped Python and typed Python that you check with strict mypy or Pyright. After using typed Python for a while, I don't think I could go back to a completely untyped codebase.
I'd probably call myself within the "dynamic typing crowd" and I've tried plenty of static typing. It mostly just slows down iterating on something, prevents issues that I/my projects don't really suffer from in the first place, and gets in the way more than it helps.
The statically typed languages I've tried are: C#, Crystal, Elm, Go, Haskell, Haxe, Java, Kotlin, Nim, Rust, TypeScript and probably more I'm forgetting about. Out of those, I've probably written most Rust code. I wouldn't say I despise static typing, but I'm not getting the same value from it that others seem to get.
I still come back to Clojure, ClojureScript or just straight up vanilla JavaScript, as they're much more effective at actually helping me solve the problem I have in my practical day-to-day.
I feel the same way about Lisps, but more strongly. It’s really fast and fun to sketch stuff up until you’re about 3 or 4 functions deep trying to figure out the shape of that one inner associative array and what made you think this little adventure was a good idea in the first place.
But that's exactly the situations where lisps shines! Select the form in your editor, evaluate it and your editor tells you exactly what it is, both runtime and compile-time data, pure magic :)
that time
but maybe not next time
Another reason I've seen people not realise how good static types are is if they aren't using a proper IDE with code intelligence.
The benefits of static typing are:
1. Fewer bugs.
2. Makes code easier to understand, because you know what types things are. That gives you a lot of information (even "business domain" things) that usually aren't documented in dynamically typed code.
3. Navigating code in an IDE that understands the language is a lot faster. E.g. you can just ctrl-click something to go to its definition, auto-complete works reliably, you can find all the uses of an item reliably, etc.
4. Refactoring code becomes tractable and easy. You can rename an item and it will automatically update all the usages. Anything you miss will get caught by the type checker.
If you don't use a proper IDE you're missing out on half of that.
Even so, I don't really see how you can say it gets in the way more than it helps. Unless you're working on really small & one-off projects, the amount of time you'll save by not having to deal with type errors or spend ages deciphering code just to figure out what type something has easily offsets any time adding the types.
Ok, does Visual Studio Code and/or Visual Studio and/or the various JetBrains IDEs count as "proper IDE"? If so, those are the editors I tried, and while they're nice and all, none of those things you listed got better compared to my non-static typed languages usage. In fact, I'd argue that some of those things get worse when using statically typed languages, especially #2 and #4.
> Even so, I don't really see how you can say it gets in the way more than it helps. Unless you're working on really small & one-off projects, the amount of time you'll save by not having to deal with type errors or spend ages deciphering code just to figure out what type something has easily offsets any time adding the types.
I don't have to spend any time figuring out what type something has because that's not a typical problem I have when reading and writing code in for example Clojure. And if I do wonder about the shape of the data or whatever, I evaluate that snippet of code in my editor and it shows me what data is inside of whatever I had selected.
It's OK that we have different ways of working and our brains work differently. Static typing is not objectively better, some things just work better in one way for some people. I really love the feedback cycle of "Read code, evaluate it, change it, evaluate it, write a test, evaluate it, save file" for producing/modifying code, and others want a cycle of "Read a lot, type a bit, run type checker, run unit tests" or whatever, and that's perfectly fine.
Of course you have to know the type of an object to manipulate it. How can you not?
Running code to see the type is pretty much the only option for dynamically typed code but it's clearly vastly inferior:
1. It takes way longer.
2. It only shows you one possibility for the type. Falls apart as soon as you have a union or optional.
> In fact, I'd argue that some of those things get worse when using statically typed languages, especially #2 and #4.
How? Static types add extra information that makes code easier to understand. In dynamically typed code it's the stuff that ends up in comments anyway, except now it's complete and correct.
Similarly they make automatic variable renaming actually work (IDEs don't have enough information to do it properly otherwise), and they detect errors when you screw up a refactor.
What's your argument for them making things worse?
Not every language requires you to know the strict type of the data to manipulate it. Take `conj` from Clojure as an example. The function adds an entry to a `collection`, so you know it's a collection, but that's all you need to know in most cases. A collection can be a map, vector, list or your own thing, but it "adds a new entry" regardless, which exact behavior depends on what you use with it.
Definitely not for everyone, but personally I like the abstracted thinking this leads to.
Besides, what I meant with my comment wasn't "I don't need to know anything" but that confusing what types I'm dealing with isn't a typical problem I have when reading/writing code. I thought I made it clear, but I suppose I could have made it clearer. That's on me.
> Running code to see the type is pretty much the only option for dynamically typed code but it's clearly vastly inferior:
Disagree, it's vastly inferior to rely on types to understand what the data actually is, and evaluating the selected form takes one keyboard shortcut and your editor shows you the actual data + type. Only being able to see the type is clearly vastly inferior to me. I like to be able to see exactly what's going on, not just being able to see kind of what's going on.
You never wanted to be able to select a function call inside of another function and be able to see exactly what that returns, without having to do anything else than selecting that piece of code and hitting a keyboard shortcut in your editor? I wouldn't want to trade being able to do that with having static types any day.
> Similarly they make automatic variable renaming actually work (IDEs don't have enough information to do it properly otherwise), and they detect errors when you screw up a refactor.
I don't know where you get the idea that automatic variable renaming doesn't work outside of statically typed languages. Just because a language isn't statically typed, doesn't mean you can't statically analyze it, see https://clojure-lsp.io/features/ for one example.
Worth repeating: It's great that you seem to be getting good value out of statically typed languages, really. But that doesn't mean it's "objectively the best" or whatever, it just means it works great for you with the tradeoffs you're willing to make. Personally, I make other tradeoffs, so other languages are better for me and what I work on. You seem to argue from the standpoint of "Obviously static typing is the best for everything, no doubt" (like many of the statically typing practitioners) but reality is almost never that black and white.
Python dicts, when used as composite types or records, are a literal hellscape. Grepping through the code to find out where stringly-keyed fields get written takes way more time than thinking about types ever would. These should be structs.
Static typing has so many advantages:
- It lowers the software defect rate. All type errors are caught for free at compile time instead of runtime. This makes the software strong and rigid instead of brittle, and it removes an entire category of tests you would have to write and maintain.
- Static typing makes code maintainable for other people, including future you. It's self-documenting. You know precisely what things are in the immediate scope.
- Static typing makes bug-free automated refactoring with tools possible. There is no greater pleasure than mutating code via its AST.
Static typing is not hard, either. Most typed languages don't require type declarations except in structs and function declarations - that's really not a lot of effort.
Optional is equivalent to no type system at all, in my opinion.
giving me some confidence is completely useless on a large scale, especially because the type system could literally be lying even when it does give me something when it is optional and unenforced (an external linter does not count as enforcement, because it is not exhaustive). I still have to write tests for everything to check types, or run some external type checker which is guaranteed to not be able to catch all errors because the language semantics don't even allow for it.
giving me full confidence is not. In typed languages, if the compiler (not some external tool) says there are no type errors, then there are no type errors. I'm not "reasonably confident". I'm sure.
Because it would be hilarious if ChatGPT was itself written in Python.
def bounding_box(points: list[Point]) -> Box:
...
I also really like being able to document the imperative code like `def do_thing() -> None`. Of course, it's completely up to the programmer to follow the rule of not doing side effects in routines that return something.
But having to do it everywhere? Ugh. I don't think people realise how powerful duck typing is for doing polymorphic code. I can write something like:
def mean(things):
return sum(things)/len(things)
But of course learning when to define static types vs when not to comes down to experience. Python treats you like an adult. I feel like a lot of people who want static typing everywhere want it as training wheels for other devs they don't trust to make the right judgement calls.
> I feel like a lot of people who want static typing everywhere want it as training wheels for other devs they don't trust to make the right judgement calls.
Now, I want it because it makes my life a lot easier. It finds many classes of errors before runtime and improves auto-completion in my editor. For most programs, it's not a lot of effort. The frustrating part is interfacing with untyped libraries while using strict type-checking, but thankfully, more and more libraries are adopting typing. The type system itself is fine: most things you want to express are doable, but it's not at the same level of complexity as Typescript. It's certainly a better type system than Go's for example.
As for `collections.abc.Sequence`, using an ABC sucks because then you have to define a class and inherit from it. I don't want to do that. I just want to pass something that conforms to the duck type. Are these now also defined as structural types/Protocols?
Duck typing is indeed powerful, but it does not need to be a runtime check.
> What am I going to do, define an ABC or typing.Protocol called `ListLike` or something?
Yes, how am I going to know what types I can pass to mean instead? I would go to the extreme of saying that a lot of python code should be type annotated (if annotated at all) with protocols instead of concrete or base classes.
Of course until type checking is properly integrated in the interpreter this is all kind of pointless.
Oh, if I'm writing it for you (ie. I'm writing a library) then I'll put documentation (probably as type annotations, as I said in the first half of my comment). But a lot of code I write is just for me, or is going to be part of a completely self-contained project and it would be obvious from context what to pass. Like I said, it's a judgement call that Python expects you to be able to make.
But I have often wondered, if someone wants static typing, why not just use a statically typed language?
A very high proportion of Python devs aren’t using it for the language, but for the libraries and ecosystem. Also lots of them weren’t the ones who picked it.
Meanwhile, not at least having the kind of autocomplete and documentation that type hints provide is kinda hellish on any project of more than 200 or so lines. The time savings from spotting runtime bugs before they happen is just a bonus.
Personally, I almost never need to add a type hint outside high-level definitions and function/method signatures, so they’re not really in the way even when I’m being pretty thorough with them.
Mind you, I prefer a type system. I'd prefer to use Typescript over Javascript, etc. But I've also used a number of dynamic languages that let me work a lot faster when needed; Tcl, Ruby, and Python are examples of these.
I've also used some type systems that lift a heavier load, letting me pay more attention to the type definitions and know that it will "just work" at the end, because, mathematically, it works. Haskell falls into this category (though I rarely use it for anything other than fun).
I get it, you haven't use a dynamic system in a way that it works out for you. But that doesn't mean they're wrong... just that others have different experiences.
If this is the first time people are introduced to static typing in programming I can understand their frustrations and opposition to it. Its probably the worst type system in any modern, popular language out there, except perhaps when things like clojure(script) pretends to have types (shudder).
So no, we're not coming around. Static typing as dictated by a heavy-handed and super-strict compiler has it's place, but has gimped our industry for decades.
However, we do have to be very careful. Static, compile-time typing has kept the hipsters and junior-devs at-bay, and kept them from causing too-much havoc as we've seen in the JS world. So it's definitely an up-coming hazard for us to navigate around and make sure we don't fall prey to. Otherwise python will turn into another JS dumpster fire. Luckily, the JS developers are too-distracted and enthralled with node.js to jump ship.
That will sadly probably never happen given the momentum they’ve built so the only choice is to retroactively add static typing and begin enforcing it in individual projects.
When starting a new project, I would suggest nobody choose a dynamically typed language. Between Swift, Kotlin, C#, Go, Rust, etc… there’s no need for anything else. As long as front end web is around you might need TypeScript - and as long as ML is Python centric you might benefit from using a bit of it. But I wouldn’t make them the primary language.
Context: Recovering dynamically typed language addict of 12 years. They’re slow, error prone, and don’t scale to a large engineering team.