Hacker News Reader

Comment by ledauphin | Hacker News Reader

this article is so full of... not-very-educated thoughts on gradual typing in a dynamic language that it's hard to know where to start.

A few concrete criticisms:

1. If you're using a dynamic language, then _by definition_ the language will not enforce your static hints at runtime. However, good news! Python has always been strongly typed, and _does_ enforce types at runtime!

2. A number of the examples given would be _impossible_ to statically type in most compiled languages (those without dependent types). It's hard to know where to go with a critique saying that you can't fully represent heterogeneous values in a dict, given that you can't do this _at all_ in many statically compiled languages.

It seems to fall back on saying "use of the type system requires too much discipline to be useful". This might be an interesting criticism on its own; however many current users of Python can say, from experience, that the required discipline is not a high enough hurdle that it prevents us from using types consistently and correctly.

There is plenty to critique about Python's static typing, but this article would have been better titled "I'm confused about Python's static type system and want somebody to show me how it's actually used in real-world scenarios".

View on HN

> not-very-educated thoughts on gradual typing

This seems like an inaccurate and condescending put-down. What is the definition of "educated" in this context? Is there a book or generally well-known resource?

The author is clearly thoughtful and curious. Near the top of the post he says "what I’m hoping for is that someone will come along and tell me that I got it all wrong. “When you do it as follows, the system works: $explanation” Because, you know, I’d really like to have static typing in Python."

> 1. If you're using a dynamic language, then _by definition_ the language will not enforce your static hints at runtime. However, good news! Python has always been strongly typed, and _does_ enforce types at runtime!

And yet, the author points out that this is a working program:

  foo: int = 'hello'
  print(foo)

In what reasonable sense can Python be said to "enforce types at runtime" here?

This one is self-explanatory.

Can a string be printed? Yes! Then Python is (correctly) allowing typesafe behavior here. Your (incorrect) annotation does not in any way contradict the typesafe-ness of printing a string (or an int). They're both equally printable.

This is just a very, very bad example, plain and simple. It 'looks' bad to a superficial reading, but in practice it demonstrates only what is already known about static typing in Python - it is enforced (or not) separately from the interpreter.

Agreed. Static typing is usually enforced during the compile stage in other languages. It's specifically meant to catch problems before running the code (hence the "static" in its name).

This said, I find the linting/type-hint-checking stage of Python cumbersome and confusing :(

It seems that your point boils down to "Python enforces types at runtime if you define 'enforce types at runtime' not to include 'enforcing that data you declare to be of a certain type actually is of that type'".

Am I the only one baffled by this way of thinking?

Python does and always has enforced types at runtime. This is called duck-typing. If you're not familiar with the concept of strong+dynamic, it is easy to see how this could be confusing. This may help. https://stackoverflow.com/questions/2351190/static-dynamic-v...

Static type annotations by definition are not enforced at runtime. This has been true of every language that has ever used static typing, including C, C++, Java, Rust, Typescript... you name it, statically typed languages only enforce their static typing at compile/analysis time.

Yes, it is possible to annotate types in Python incorrectly. It's possible to do this in all other languages that allow type-unsafe behavior (whether natively or via reflection, etc). This may be less common in some languages than in others, but it is fundamentally possible in the vast majority of languages that perform static typing, because those types are fundamentally enforced at analysis time, not runtime.

The author of the article seems to be unfamiliar with these distinctions, and maybe you are too. It's fair to complain that these distinctions are confusing and make things harder for programmers. Nevertheless, very few languages have ever been designed with a type system that acts exactly the same at analysis time and runtime. It's a very difficult problem, partly because these are all abstractions from the perspective of the underlying computer, which only understands boolean logic and integer/floating point math, and has virtually no other notion of types in any formal sense.

Type systems are a complex topic, and as someone who has been paying attention to them for a while, it's frustrating to see uninformed discussion of them show up on Hacker News. That said, it's perfectly understandable that people are confused by these distinctions, and rest assured there's a lot of effort going into developing better languages that suffer _less_ from these issues.

For the everyday working programmer, however, there are currently lots of tradeoffs to be made, and Python's approach to static+dynamic typing is actually pretty usable, all things considered, which is why the community overall has embraced the new static typing despite its blemishes.

> Yes, it is possible to annotate types in Python incorrectly. It's possible to do this in all other languages that allow type-unsafe behavior (whether natively or via reflection, etc). This may be less common in some languages than in others, but it is fundamentally possible in the vast majority of languages that perform static typing, because those types are fundamentally enforced at analysis time, not runtime.

It is pretty damn hard to make this mistake in languages that enforce static typing. I mean, you would have to go out of your way to do so. In Python, however, it is trivial to write the wrong type signature or to modify the body of a function so that it no longer matches the signature.

"Fundamentally possible but terribly unlikely, as opposed to the Python way of doing it" would be a more accurate description.

I think your (and the article's) argument could be summarized as "static type annotations without automated enforcement by actually running a type checker considered harmful".

Since this argument is not meaningfully different from "comments that are lies considered harmful", it seems fair to expect reasonable people to dismiss it as uninteresting.

> it seems fair to expect reasonable people to dismiss it as uninteresting.

Do you think that's a fair treatment of someone who disagrees with you but has been respectful of your opinion so far?

I realize proglang debates are flamewar territory. But I have been very careful to state I find Python type hints puzzling and less useful than they should be. It is obviously an interesting opinion shared by many others, not just me or the article's author.

As a fan of statically typed languages, I do indeed find some Python programmers engage in a form of Stockholm's syndrome. It usually takes the form of the assertion "I never found a bug that was a type error". Have you or anyone you know ever said this?

In my mind, the opposite statement of the one you attribute to me would be "wishful thinking and a positive attitude is enough to catch mistakes, it's not necessary to have help from automated tooling". In this day and age, I cannot disagree more.

I'm very confused by your response. Where did I suggest that I don't like type systems or think automated tooling is not useful?

It's precisely because I do like and use them that I find Python's static type hints to be extremely useful for large Python codebases.

What I hear you saying is "Python does not offer these things". What I hope you hear me saying is "Python does, in fact, offer these things".

Well, you did frame the discussion as:

> I think your (and the article's) argument could be summarized as "static type annotations without automated enforcement by actually running a type checker considered harmful"

It illustrates your mindset, in my opinion. I think type annotations without checking them (or without a satisfying implementation of said type checking) are mostly pointless.

You also left out the part where I remarked on the dismissive tone of your reply.

It is meaningfully different because, as the article says, comments are known to not be machine verified and thus to potentially be wrong. With type hints, they’re usually verified, but not always, so you’re more likely to trust them and get a nasty surprise when they turn out to be wrong.

However, there are other aspects to the article’s argument, such as the fact that it’s easy to end up with type annotations going silently unenforced even if you do run the checker.

Duck typing is really enforcement of interfaces, not types themselves. Everyone who has passed a string into a Python method expecting a list of strings has run into this. It works, but did you really want to process a character at a time?

Also, some languages like Java do actually enforce types at runtime. If you've ever called a Java method dynamically with the wrong types, you'd know. You can run into ClassCastException, or worse (ClassNotFoundException if you have the wrong classloader.) This is a runtime error.

> Duck typing is really enforcement of interfaces, not types themselves

A type system is just a set of rules that must be followed. The rules can be as weird or as simple as you want. Just because <int> + <string> is allowed doesn't mean it's not enforcing types.

Not really a criticism, just trying to push to expand your idea of what a type system is.

No, you're not the only one. All of these gymnastics to justify the behavior of the string-declared-int baffle me.

Is it "legally" baffling, i.e., do I understand why this behaves in the way it does purely mechanically, and do I understand the arguments the gymnasts are making? Yes.

But from an idiomatic, colloquial, linguistic, syntactic, programming-historical, or programming-cultural standpoint? Baffled.

Even calling it gradual typing is baffling. It's just adding metadata-that-doesn't-look-like-metadata that sophisticated programs that aren't part of the Python bytecode compiler can use. The definition I know is the same one from Wikipedia:

> Gradual typing is a type system in which some variables and expressions may be given types and the correctness of the typing is checked at compile time [...]

This is definitely not what Python is doing.

> This is definitely not what Python is doing.

Maybe not by default but there is tooling that makes it work that way — that’s exactly how the python build system works at my company. I don’t know the details but when I “build” python code (creating bytecode pyc files) it produces compiler errors and fails to build if I e.g. try to pass an int to an f(x: str). This has usefully caught bugs before I ran some expensive/time-consuming scripts.

I guess you could say this doesn’t count as it’s a separate program doing the type checking but IMO there’s not a clear distinction between that and e.g. having a first type checking pass in the “same” compiler program.

I don't know the latest but PEP 484 writes:

> While these [type] annotations are available at runtime through the usual __annotations__ attribute, no type checking happens at runtime. Instead, the proposal assumes the existence of a separate off-line type checker which users can run over their source code voluntarily.

Right, it’s definitely still a voluntary feature. What I meant is if you opt in and make it part of your project’s standard build procedure for python code, you effectively get the behavior and benefits of “compile-time type checking” (to the extent that you actually use type annotations in your code).

>> Gradual typing is a type system in which some variables and expressions may be given types and the correctness of the typing is checked at compile time [...] > This is definitely not what Python is doing.

When I read "compile time", I mentally replace it with "before runtime". There isn't a real compilation step in Python, so it's reasonable to accept static code analysis as part of "compile time" in that definition.

The point is that type annotations will let a static analysis tool make sure that the program is correctly typed. This can be enabled easily in most of the modern IDEs and editors. You could just decide to block the code's deployment in CI/CD if the type checker raises issues, for example.

This is how all or most statically-typed languages work, too. The typing is purely a compile-time construct.

E.g. this kind of code: https://play.rust-lang.org/?version=stable&mode=debug&editio...

typically works fine (or worse, works fine in debug mode and causes unpredictable UB-related bugs in release mode).

std::mem::transmute here is a no-op at runtime; all it does is subvert the static type checker.

The difference between statically-typed languages and things like Python with type hints is _not_ that the former has any runtime type checking; it’s that subverting the (static!) type system is more difficult and requires more obvious ceremony, and that static type checking is required whereas with Python it’s optional.

If you hacked rustc to disable static typechecking you would get a similar experience to Python with type hints but without any mypy-like tools. (Of course, this would be hard to do in practice, because Rust, like most statically-typed languages, uses types not just for typechecking but also for code generation and method dispatch.)

I agree with most of your comment, but I think there's a miscommunication problem here.

When Python "enforces types at runtime" this is the actual types, not type hints. Type hints are not a runtime artifact. The "actual type" here is "string", and enforcing it means Python would not allow invalid operations on it without error'ing. A programming language that will let you do basically anything to any value because it doesn't enforce type safety is of course C. Python is safer than C.

Yes and no. C will let you do things like:

  *(unsigned long*)0xFFFFFF14 = 0x749235f8;

It will not let you do things like:

  *0xFFFFFF14 = 0x749235f8;

   char* s = "abc";
   *(unsigned long*)0xFFFFFF14 = s;

It also won't let you call thing.method without thing.method definitely existing.

Best of all, all of these are enforced at compile time.

Now, C absolutely will let you convert an int to a pointer, or a char to an int, or an array to a pointer, or... well, it will let you convert lots of things to lots of things. Some of those things are unsafe unless you are quite sure what you're doing.

> A programming language that will let you do basically anything to any value because it doesn't enforce type safety is of course C.

False on several grounds. You can't call something that isn't a function. You can't call a function on something that doesn't have it. You can't use an int as a pointer or an array without casting it. You can't use an int as a dict (not that C has any built-in idea of what a dict is...)

> Python is safer than C.

Depends on what you're doing, and what kinds of errors you are more prone to.

I find this level of nitpicking puzzling.

You know what I meant and what error I was clarifying for the comment I was replying to. Yet you went out of your way to point all sorts of irrelevant mistakes in my post, when the gist of it was right.

Do you feel it was more important to correct me on these trivial things, or was my reply more or less correct when fixing the conceptual error in this sentence?:

> "In what reasonable sense can Python be said to "enforce types at runtime" here?"

Here's a nitpick of my own to your post:

> "Some of those things are unsafe unless you are quite sure what you're doing."

Wrong. Type (un)safety does not depend on you "being quite sure of what you're doing".

PS: assume I know C and that I've programmed complex things using it. No need to catch yourself with statements like "not that C has any idea of what a dict is". Assume we are both C programmers. It will make you sound more polite.

1. You might need to re-read the site guidelines about assuming good faith.

2. Your last two sentences were, I think, not needed for the point you were making. They were also somewhere between gross generalizations and flat-out wrong, and felt like a gratuitous slam on a language that wasn't even the topic. I thought that deserved a response, even if it wasn't your main point.

3. I may assume that you know C. I don't assume that everyone reading this exchange knows C.

> "They were also somewhere between gross generalizations"

Pointless correction. Gross generalizations are not wrong in this particular situation, which is not meant to teach anyone C.

Let me try again.

Do you think that your correction was more important than helping the poster of this comment correct their conceptual mistake?

> "In what reasonable sense can Python be said to "enforce types at runtime" here?"

All true but all rather irrelevant to their point

> You can't call something that isn't a function.

Sure you can. That's exactly what you do when you call a function returned by `dlopen` -- it might be a function, but it might also just be garbage. It might even be garbage that's coincidentally marked as executable and does some stuff before eventually crashing!

> You can't call a function on something that doesn't have it.

This is also very easy to do -- you can default-initialize a structure containing function pointers, and then call one of them. You'll be calling some random crap on the stack, which might or might not do anything of interest. But either way, C will happily let you do it.

C isn't completely untyped (there are, as you've observed, things it will forbid at compile time), but it's about as weak as a static typing system can be.

I think you're talking past the comment you're replying to.

In both examples, you have tell the compiler exactly what the types are. In the first one you have to cast the return value of `dlsym` to a function pointer, and at that point, as far as the compiler is concerned, it's a function. In the second example you have to explicitly declare the struct containing function pointers, so of course you can call its members.

So it's strange to suggest that C is somewhat untyped; it's not untyped at all. C just makes it very easy to force the compiler to assume different type for a value (that's necessary for low-level code like the examples in the comment you replied to; and of course that makes it very easy to do unsafe things, I don't think anyone disputes that).

And about the "weak typing system" comment, note that nobody agrees on what exactly that means. Just as an example, see how this page[1] defines "weaker" vs "stronger" types, and how it's completely different from the way you're using here. In general, I find people call type systems that don't do what they expect or want "weak", but that's not an useful discriminator: at that point you might just call it "bad typing" to dispel any illusion that it has any objective meaning.

[1] http://book.realworldhaskell.org/read/types-and-functions.ht...

I didn't say it was "somewhat untyped." I said it has a very weak static typing system.

Just because "weak typing" means something different in Haskell doesn't mean it can't be used meaningfully (and beyond "bad") in the context of C. C's typing is historically referred to as "weak" because C's notion of casting doesn't distinguish between type and value transmutation: pointer-to-pointer casts don't convert the referent (because they can't), which in turn gives the C compiler very little leeway in proving that the program's types as declared have any particular meaning at runtime.

Compare this to Python, which is "strongly" typed in the same sense: doing `y = str(x)` on `x` means that `type(y)` actually is `str`, and not merely a promise to the runtime. It's enforced, which makes it strong.

My point about "weak typing" is this: if someone says "hey, I'm designing a new language, its type system is going to be weak", they gave you no information whatsoever about the type system other than maybe "some people on the Internet will probably complain about it".

That's why it's useless to say things like "it's about as weak as a static typing system can be" and "has been historically considered weak".

If you had explained your objection to C's type system (like you have now), I wouldn't have said anything. And I've been on the Internet long enough to have seen C's typing is historically referred to as "weak" because X for *many* values of X, so I disagree that that's the only or even main objection.

For 1, there's some unfortunate but important semantics. Python does not enforce type _Hints_ at runtime, as Hints are in many ways fancy comments. But as pointed out, python is a strongly typed language (at least on the sliding scale of strong to weak), and types are known at runtime (call `print(type(var))` to see them). And calling `1 + 'a'` will result in a TypeError exception, unlike say JavaScript.

I believe this is the relavent passage in the article the GP is referencing, which is incorrect if you take "runtime" to mean something like when the line is executed: "Python is a dynamically typed language. By definition, you don’t know the real types of variables at runtime."

It _is_ however correct to say you don't know the real type _before_ runtime, at least python does not.

I think the top comment was less polite than it should be, but I would echo that the post seems to have a bit of a weird understanding of gradual typing?

The Any type is to allow incremental typing. You start out with dynamic code, and progressively 'typify' it by adding more and more annotations, with Any working as a stopgap where for the moment no valid type exists, until you finally have a fully typed program. Inserting Anys because you're lazy is like casting things to Object in Java because you're lazy, abusing it like that is just incredibly sloppy and bad code.

If someone is really struggling with this, simply use a linting rule to ban Any in 'mature' code.

Like the author said, the weird thing is that it goes both ways. I'm pretty sure if you try to return `any` from a function with an explicit return type in TypeScript you'll get pinged for it.

Not with any TSC settings I've ever used.

noImplicitAny is a very common config option that helps catch a ton of bugs. I wouldn’t use TS without it.

noImplicitAny is different than what's being discussed (I think). Returning something typed as `any` in a function that has a different return type is totally fine: it's not implicit (you've cast it to any), and it passes (you're saying the type is literally anything after all).

There are lint rules to avoid typing as `any`.

There are "no explicit any" linting rules as well, which are a must.

Python enforces its runtime types at runtime. 'hello' is a perfectly valid argument to print. Python will raise a TypeError if you try to use it to index a list though.

> In what reasonable sense can Python be said to "enforce types at runtime" here?

Python is both strongly typed and protocol-typed. In the case of `print(...)`, the protocol is that the received parameter responds to `str(...)` (i.e., has `__str__`).

(What Python isn't is statically typed. But "strong" and "static" are completely different typing dimensions.)

I think the person you're responding to means to ask

> In what reasonable sense can Python be said to "enforce statically declared types at runtime" here?

which is certainly an extremely pertinent question for a language that allows declaring supposedly static qualities of a program.

> In what reasonable sense can Python be said to "enforce statically declared types at runtime" here?

It can't, because Python doesn't have statically declared types. It's a strong, dynamically typed language.

Python also doesn't allow you to declare static types. It allows you to declare type hints, which are a sort of adjacent and potentially unrealistic (or just plain incorrect) typechecking proof that doesn't correspond at all to runtime behavior.

If your statement were absolutely true then a product like mypy would neither exist nor provide value.

Python has the capacity to have some form of gradual typing built-in at the language level, but it does not, which has led to the enormous confusion we have before us. The SBCL implementation of Common Lisp is a great example of a strong, dynamically typed language that is able to, when possible, detect errors statically at compile time. That tech is 30 years old now.

There is no reason why the Python bytecode compiler could not statically or dynamically detect this programmer error at either compile-time or run-time present in this statement:

    x: int = "x"

Compare to Common Lisp, well known for being dynamically typed (so dynamic that it's possible to change the class of an object at run-time!):

    (declaim (type integer x))
    (defvar x "oops")
    
    ; ==>
    
    Unhandled SIMPLE-TYPE-ERROR in thread #<SB-THREAD:THREAD "main thread" RUNNING {10005D05B3}>:
      Cannot set SYMBOL-VALUE of X to "oops", not of type INTEGER.
    quitting
     
    compilation unit aborted
       caught 1 fatal ERROR condition

Instead, this job in Python is haphazardly relegated to the program readers and IDE implementers.

> If your statement were absolutely true then a product like mypy would neither exist nor provide value.

I don't see why that would be the case. The value of Mypy is that it's essentially a proof language over something that resembles Python[1]: it's a way to do some amount of static typing without having to maintain a separate copy and representation of the program.

And sure: Python could detect inconsistent type hints at runtime. But why would it? It's already strongly typed, and the presence of type hints usually means that someone is already running Mypy or another checker during development. It's not clear that there's a significant advantage to be gained, particularly one that justifies the additional overhead.

[1]: "Resembles Python" because mypy does not actually evaluate any Python. It doesn't know what types your program has at runtime; it only knows type hints and a few small rules (for things like string literals) and trusts the developer to reconcile those rules with Python's runtime behavior.

> Python could detect inconsistent type hints at runtime. But why would it?

Because the programmer explicitly intended a contract which is being violated, which would in turn indicate a bug.

Do you think other languages should do the same, because very few do any kind of runtime type checking (e.g. bounds checking) due to the performance penalty. Why focus specifically on python's lack of runtime type checking?

Anecdotally, my code in python is functionally fully typechecked and runtime type checking would solely penalize me by slowing down my code. There's no need for it because the type checker correctly proves the soundness of the program's types, in exactly the same way that Java's or C++'s do.

The point is precisely that you don't need to suffer the costs of runtime type checking if you have a solid typechecker. Lisp's approach here is worse than the one adopted by python/js/ts/go/etc.

What other languages do that though? C doesn't, Java doesn't outside of particular cases. If you can trick or mislead the compiler, the runtime often has no type information at all, so you'll get, at best, a ClassCastException, and more often than not get a segfault.

Python can't be said to enforce types at runtime. AFAIK, it doesn't say that. In fact, the original article starts off with "I'm expecting to be able to run a static analyzer", then it goes on to say "This isn't caught at runtime".

Yes, but is it caught at static analyzer run? Yes, I typed it into vi and ran mypy on it and it said "error: Incompatible types in assignment (expression has type "str", variable has type "int")."

But I didn't have to run mypy on it, my editor told me: "Expression of type "Literal['hello']" cannot be assigned to declared type "int"".

Maybe the word "educated" wasn't the right choice in this case, can you offer a better one? I honestly couldn't make it through the entire article, despite several attempts. It is full of inconsistencies and what seem like wilful misunderstandings to justify the authors conclusion.

Particularly as the author starts off saying "run a static analysis tool" as if knowing that's how you go about it, and then saying "it doesn't catch it at runtime".

> Python can't be said to enforce types at runtime.

It does: in form of TypeError exceptions. It doesn't enforce type hints, sure, but that's why they're called hints and not declarations.

> In what reasonable sense can Python be said to "enforce types at runtime" here?

    In [1]: foo: int = 'hello'

    In [2]: foo + 1

    TypeError
    Input In [2], in <cell line: 1>()
    ----> 1 foo + 1

    TypeError: can only concatenate str (not "int") to str

Exactly right, great example. And furthermore:

    >>> foo: int = 'hello'
    >>> type(foo)
    <class 'str'>

Python's type annotations are annotations. Types are being enforced at runtime, it's just that some are thinking that annotations have some effect on the interpreter, when they do not.

This is a demonstration that it has not enforced what many would reasonably believe is a static type declaration.

But we have (to quote another commenter) "uneducated" people making a supposedly boneheaded mistake to assume

    x: T = y

is a declaration that x will only contain values of type T, or otherwise something will be detected by cpython as invalid (at either bytecode-compile-time or run-time).

Instead, this line is more properly interpreted as "documentation reasonably believed to represent a type saying something about values that x can be assigned at runtime, that's not in a docstring, that some programs may query."

I can totally forgive anybody who believes the former over the latter, and expresses their belief in a statement like that which you've quoted.

Type hints are not static type declarations. Python does not refer to them as such, nor does Mypy or any other typechecker for Python type hints.

Absent of active machine checking (e.g., via mypy), type hints should be considered exactly what you said: another form of documentation, one that might just be wrong.

This is my point precisely: it looks like a static type declaration to anybody who has seen what a static type declaration looks like in any statically or gradually typed program in the last 50 years.

But in Python, it's not (and documented in a PEP as not), and it causes massive confusion.

That's the way it works in Ruby with RBS, and it will work the same way in JavaScript should Microsoft's proposal for type hints make it into ECMAScript[1].

[1] https://devblogs.microsoft.com/typescript/a-proposal-for-typ...

Yeah, you are talking about actual types and not type hints. It gets confusing. Python doesn't enforce type hints at runtime.

Don't confuse strong typing and static typing.

Python is strongly typed. It respects types at runtime e.g., `"1"+1` causes TypeError. Python is not statically typed.

For comparison, C is statically typed but it is weakly typed e.g., printf function has no idea about actual types of its arguments (in particular, errors in the format arg may produce non-sensical results silently).

The "print" function will take any Python object and call its "__str__" method. Not sure if it will do it to a string, but the outcome would be the same.

As far as I remember, the "__str__" method is present on all objects in some form, so print will happily take any type.

Come on. It’s uneducated because it’s just a naive hot take. The education in this case comes from experience and basic understanding of the underlying issues. The author does not demonstrate possessing either of these attributes.

Personally, I took exception to the following from TFA:

> Python is a dynamically typed language. By definition, you don’t know the real types of variables at runtime. This is a feature.

In "you don’t know the real types of variables at runtime", this is just flat wrong. One doesn't know the real type of a variable until runtime. It seems to me the author maybe has a bit of confusion between weak typing and dynamic typing here.

I get what they mean though. In a typed language you know the type of a variable at compile time. In dynamically typed languages there is nothing to enforce this and a variable can be passed with any type. So there wording may be off but the point stands.

> "you don’t know the real types of variables at runtime", this is just flat wrong. One doesn't know the real type of a variable until runtime.

I read the sentence as

  "you don't know the real types of variable at [the start of] runtime"

as opposed to

  "you don't know the real types of variable *during* runtimes"

So I understood the author to mean what you said. Could have been clearer you're right.

In fact that quote is almost exactly the opposite of the truth. “Dynamic” literally means “at runtime”.

> One doesn't know the real type of a variable until runtime.

In the program

    def x(f):
        return f(3)

what is the "real type" of `f` at runtime; say, at the instant just before the call to `f`? How can you tell?

If your answer is that `f` has type "function-like", then that's a much weaker kind of a type than even type hints offer, let alone a real type system.

> what is the "real type" of `f` at runtime; say, at the instant just before the call to `f`? How can you tell?

We can tell by looking at what python triggers TypeErrors for.

>If your answer is that `f` has type "function-like", then that's a much weaker kind of a type than even type hints offer, let alone a real type system.

f implements the Callable interface - meaning it's either a function/method or an instance of a class with a `__call__` method.

So you can pass e.g. `print` or an object with a `__call__` method. If you try to do `x(5)` that's an immediate runtime TypeError because `5` can't be called.

f must also accept one int argument. It can have additional optional arguments, and it can also accept other types (e.g. `f("foo")` can also work!), but it must work on one int. The one int can already be optional - the function can also work with zero arguments.

Calling e.g. `"foo".join(5)` fails with a TypeError because 5 isn't Iterable, and calling `"foo".join([1], [2])` fails with a TypeError because it gets more arguments than expected.

Right, "function-like". I thought OP was meaning the static type of a variable, which genuinely can't be known fully at runtime in general. But it turns out that they meant to write "... until runtime", not "... at runtime", which makes this all a bit moot. And invites more confusion about the two disjoint usages of the word "type" in the PL community, as seen liberally spattered about elsethread, but that can't be helped :-)

So PHP is not a dynamically typed programming language according to you? It enforces type hints at runtime just fine.

There is nothing in the definition of dynamically typed languages that says they can't use type hints to perform runtime checks.

Was going to say the same for some implementations of Common Lisp, notably SBCL, which by default treats type declarations additionally as assertions. Subject to limitations it can use declared types and infer more types at compile time to produce more optimal assembly and give warnings if it detects type errors, or interestingly warnings if it has a chance to use e.g. a fast assembly add on a fixed sized int if you help narrow its inferred type with an explicit declaration one way or another. But unless it can prove a type always holds, it will also by default check declared types at runtime.

I can only speak about my own experience:

Most people I know writing Python don't use type hints because they are too much of a hurdle for very little payoff. The tooling that pays attention to type hints is slow as molasses or difficult to use or understand. The type hints themselves are of dubious use.

As a fan and advocate of static typing, I find myself advocating for type hints anyway, but I must agree I often can't reply anything to my coworkers' objections, because Python type hints are truly not that useful.

> The tooling that pays attention to type hints is slow as molasses or difficult to use or understand.

I use mypy daily on big codebases, it is fine, not fast, but fine.

> The type hints themselves are of dubious use.

They tell you when you make type errors, they help you understand what type things should be. The same as in literally every other language with static typing. There is nothing special here, nothing different. python with a static type checker running in strict mode is not fundementally different from Java's static type checking.

> As a fan and advocate of static typing, I find myself advocating for type hints anyway, but I must agree I often can't reply anything to my coworkers' objections, because Python type hints are truly not that useful.

What do they lack that would make them useful?

> The same as in literally every other language with static typing

No, obviously not the same, otherwise I wouldn't be complaining. They are not even on par with Typescript, which I'm not a fan of either.

> [type hints] tell you when you make type errors

Not according to other comments I seem to be getting here. Other people are arguing type hints are not primarily for checking, but a form of notation for documentation. Seems wasteful, and I wish the Python community and tooling decided instead that they are for actually checking them.

> What do they lack that would make them useful?

Standardized, go-to tools that work in the build pipeline and that catch most errors without taking a long time to do so.

I haven't found mypy to fill these requirements. It's so bad I cannot convince my coworkers to make the effort to write more type hints.

> No, obviously not the same, otherwise I wouldn't be complaining.

What is the difference?

> They are not even on par with Typescript, which I'm not a fan of either.

Go is not on par with Typescript or Python, I still don't think it is okay for people to just say fuckit and `interface{}` it all and it is still shit to work with code that does use `interface{}`. At least Python with mypy has null safety, something that Java and Go does not have. There are some places it is worse than other statically typed languages, others where it is better.

> Standardized, go-to tools that work in the build pipeline and that catch most errors without taking a long time to do so.

It is mypy. What actual type errors does mypy not catch that you want it to catch? Why can't you use it in the build pipeline? I do it every day, it catches all the errors it should. The one complaint I can maybe see is the duct type compatibility complaint, and it is not really something that comes up that often for me, and definitely not something I would say invalidates the whole concept.

> What is the difference?

I'm not going to repeat myself, I already told you.

I find mypy slow, unsatisfying, inconsistent, and it fails to catch many type errors. No, I'm not going to go look in my work laptop to give you an example.

> There are some places it is worse than other statically typed languages, others where it is better.

In most places it is way worse, and I'll find it very hard to find common ground with anyone who disagrees on this.

Feel free to disagree, but I don't find this conversation useful.

How big is your code base? I’ve worked with really large auto generated codes and mypy never took more than a minute or two to run. On more normal sized code bases it’s just a few seconds. It’s also incremental so subsequent runs are faster.

Compare that to a c++ project where it’s not unusual to have 10, 15, 30 min build times.

Or even worse, the time to wait for discovering the problem runtime in prod.

The type hints also help in IDE navigation, where there is zero time to wait, pycharm can go to definition or find references in under a second.

Better code completion and documentation in you editor is another great feature of type hinting

Have you looked at the tools recently? LSP + Pyright is very fast and plugins exist for many editors. Or maybe it's slow on larger codebases or very large files? I don't have that broad an experience yet, but so far it's been very good.

Sorry, I wasn't clear: I want command-line tooling for the build pipeline, not for the IDE or any individual developer.

mypy is quite fast at the command line. certainly fast enough for CI usage, where speed is generally less critical than in an IDE.

Not my experience!

What speed are you looking for in your CI pipline that you are not getting from mypy? Perhaps you are working on a much larger codebase than I am but the app I use it in takes under a minute. And all the unit tests take longer to run.

A 25k LOC Django project - takes 15-20 minutes to run mypy... Unit tests take about 3 minutes.

That's locally on a fast (ish) laptop. GH CI takes 10 minutes to run tests, so I'm a little hesitant to see how long mypy takes to run.

Use "dmypy", it is fast.

pyright is a command line program https://github.com/microsoft/pyright

in my experience it's a far better type checker than mypy, which tends to silently not check things without you ever realising

Thanks for the suggestion! I didn't know this checker existed. Will definitely try it.

> 1. If you're using a dynamic language, then _by definition_ the language will not enforce your static hints at runtime.

Counterexample: PHP is a dynamic language which enforces static type declarations at runtime.

Meh. you can do the same thing in Python if you really want - there are dynamic interpreter shims that can do this.

The point is that it is not expected by definition, since by definition, static != runtime.

You're assuming that type hints are by definition static-only. There's no particular reason why type hints can't be both statically and dynamically enforceable, as they are in PHP.

Fun fact: it will still do type coercion in that scenario without strict mode. And you cannot annotate assignments. Otherwise it’s pretty useful.

It seems to me like Python needs to get Typescript's capabilities (if Python wants to go further this way of course). It solves all these problems very well, and has no problem with 2 and most of the author's objections.

Typescript does absolutely fantastic type inference, which mypy definitely does not. I think that's a huge advantage for Typescript over mypy, and it's really foundational to the value it provides.

However, even in Typescript it is nontrivial to _annotate_ these complex types, and in most cases it's still possible to do in Python - you just need to commit to using something like dataclasses rather than pretending that what you have is a `dict`.

Basically, to get the most out of static typing in the world of Python, you do have to write more boilerplate than you would in Typescript, and in particular you need to avoid dicts and prefer various sorts of class-based data containers (again, dataclasses, attrs, namedtuples, etc).

> However, even in Typescript it is nontrivial to _annotate_ these complex types

The only one which stands out to me as non-trivial in TypeScript is thrown exceptions… and even then only because you can only type them in a catch clause, and only with unknown, and they’re invisible to callers. But then this is why people who would care for checked exceptions quite reasonably just use something like a Result type.

But I agree with your conclusion completely, and it’s basically the same one I came to reading the post.

I was curious about how the current state of things compares to TS, as I may soon have a foray back into Python soon, and would be delighted to bring some static types into the mix. Pretty much as I expected, it’ll probably be more effort/less valuable than TS, but more valuable to me than to the author.

I had to come back to Python after working with TypeScript for a while. After going from Python 2.x to TypeScript and being very impressed with static typing, I didn’t want to go back to untyped so I had to get up to speed with the ecosystem. My findings:

— Consider using Pydantic (either its drop-in dataclasses replacement, or its models). It has issues and documentation is lacking in places, but in many use cases it’s a boon when it comes to being confident about data you read and/or pass around.

— Don’t use TypedDict if you foresee needing anything like optional keys, especially with defaults. I went with TypedDict and it came back to bite me later (though was relatively easy to refactor).

— Keep an eye on https://pypi.org/project/typing-extensions/, it has some useful utility types. I wish I knew about this package a couple of months back.

— If you use VS Code (which I started after switching to TS), there’re some shenanigans to be aware of. Unlike TypeScript’s elegant approach where you’re in control of installing typing versions, VS Code’s official Python extension will force-install unvetted third-party typings, not caring if they don’t match library versions in your use and not allowing to opt out. In later versions you can unset mypy in Python extension settings, and use a separate Mypy extension that doesn’t do this behind your back and doesn’t send you deep into some rando’s typings when you hit F12 to jump to definition.

— That aside, VS Code can provide development experience is not that far from TypeScript. You can keep a virtualenv nearby and select its Python bin as interpreter in your workspace, giving Mypy access to dependencies (and if VS Code’s interpreter selection GUI resolves symlinks, you can enter interpreter path by hand so that it doesn‘t).

— Sphinx documentation is good at picking up typing hints and works reasonably well with Pydantic’s models, too.

Thank you for so much detailed advice! I’m sure this will be valuable to reference as I dive in

You can type Python dicts, see typing.TypedDict. It’s still lacking though, totality is all or nothing until py311, that is to say, TypedDict’s are either Required<T> or Partial<T> in TypeScript parlance. And you don’t get Pick<T>, Omit<T>, etc.

The ability to slowly add typed code to an existing database has been a lifesaver. I always hated JS because of callback hell (and the notions around its community) but since I started working with typescript my mind has been changed. I love it

> If you're using a dynamic language, then _by definition_ the language will not enforce your static hints at runtime.

I don't think this follows by definition. It's common not to check the static hints at runtime, but a dynamic language could choose to treat them as contracts or assertions. Over in Lisp-land, SBCL will enforce static type declarations at runtime [1], contrary to most other Lisp compilers, at least under the default compilation settings (you can force it to skip the checks by compiling with a low "safety" level). If the compiler can prove that a static type declaration always holds, it will omit the runtime check; otherwise it will compile it into a runtime assertion.

[1] http://www.sbcl.org/manual/#Declarations-as-Assertions

> If you're using a dynamic language, then _by definition_ the language will not enforce your static hints at runtime

What definition is that? Raku is a dynamically language with gradual typing which will enforce those types at runtime (if it cannot do so at compile time).

> you can't do this _at all_ in many statically compiled languages

What statically compiled language doesn't provide a way (sometimes clunky) to have heterogeneous values in a dict? You can do that with void* in C, even

> can't fully represent heterogeneous values in a dict, given that you can't do this _at all_ in many statically compiled languages.

This isn't really true. For example, in Java it's Map<K,Object>. For languages without subtyping, existential types are often used. It's quite common to see in infrastructural code, e.g. caching.

That’s the point though:

Duck typing and heterogeneous dictionaries are (and have been) standard Python.

Adding a type system which doesn’t respect duck typing by trying to access the member even when the types don’t match or which can’t express standard idioms used in Python seems a poor architectural choice.

It’s not saying that Python’s type system is “too much discipline”, but rather that the type system doesn’t encode typical Pythonisms.

Python typing supports duck typing: https://peps.python.org/pep-0544/

and heterogeneously typed ducts: https://peps.python.org/pep-0589/

And you can make it work at runtime too! So as long as you have the right methods and attrs isinstance on your duck-typed object will return true!

I read 0544 and the proposal for protocols fails to support the main benefit of duck typing, as it requires the substituted-for class to be defined as a protocol.

Traditionally, duck typing is used to inject types into a library that isn’t expecting extension at that particular point — eg, substituting a test class for a real class in a data object that normally wouldn’t be a protocol.

I’m not seeing how that PEP addresses that use case.

- - - -

Similarly, the heading on the dictionary says “for a fixed set of keys” — but what if I want a dynamic heterodox dict? Eg, unpacking JSON.

You just end up shoving “any” all over the place. At which point, are the types helping?

Though, protocols do help the dict case for typing:

dict : Hashable -> Any

Can you re-read please?

https://peps.python.org/pep-0544/#:~:text=Structural%20subty....

> substituted-for class to be defined as a protocol.

... which is false

> Similarly, the heading on the dictionary says “for a fixed set of keys” — but what if I want a dynamic heterodox dict? Eg, unpacking JSON.

You can quite obviously decode to recursive types (e.g. using pydantic), not sure what the problem is.

Your link appears not to work (for me) — can you cite what you believe I have incorrect?

This section seems to agree with me, where it explains why normal classes can’t be subclassed to protocols:

> Now, C is a subtype of Proto, and Proto is a subtype of Base. But C cannot be a subtype of Base (since the latter is not a protocol). This situation would be really weird. In addition, there is an ambiguity about whether attributes of Base should become protocol members of Proto.

https://peps.python.org/pep-0544/#protocols-subclassing-norm...

- - - -

I’m not sure why you think it’s “obvious” that you can use a third party library to solve the problem — or why that addresses my complaint that the built-in type system doesn’t work for that.

If anything, the existence of a third party library hints the standard library doesn’t cover the use case.

No offense, but have you actually given it a shot? Duck typing is supported and you can get virtually all use cases of heterogenous dictionaries by using union types and duck typing. The article even touches on this.

edit: To be more precise: The article specifically mentions that accessing invalid members give you a type error, but posits that people will probably not bother and just use 'any' instead. How is that not saying 'too much discipline'?

Yep — I use mixed annotations on my Python code because as the person I’m responding to pointed out, they catch many small type errors. Dataclasses have been awesome.

I’m also generally pro-types, but I think it’s worth having a discussion about this type system in the context of Pythonisms.

- - - - -

If you’re using “any” for most of your types, then it’s not providing value — an untyped statement implicitly has the type “any”.

There’s a reason I brought up the JSON example: loading and manipulating JSON of varying structure is something I do a lot at work.

Sure, but the idea behind gradual typing is that 'Any' should only be temporary. IMO if you're serious about it, stable code should at least pass strict type checking, if not completely forbid even explicit Any.

>There’s a reason I brought up the JSON example: loading and manipulating JSON of varying structure is something I do a lot at work.

Do you have a specific example that you find troublesome? For JSON with a fixed, regular structure (i.e. no subtyping) something like Pydantic works well, otherwise using unions + protocols with custom validation code has covered even my most esoteric use cases so far.

If we're talking about unstable JSON, a recursive union type works well enough in my experience, though defining that type gets a bit ugly if your type checker doesn't directly support recursive types.

Your suggestion to “use a recursive union type” and that being “ugly … if your type checker doesn’t … support recursive types” being required to support varying JSON is my point:

Python types don’t support a common Pythonism used frequently in my work.

In fact, you admit that even fixed JSON can be difficult without a 3rd party library.

You’re lecturing me like I don’t understand types without realizing that you repeated my point about a gap in the current type system.

I think I might not have been clear: this is not a limitation of the type system, but of a specific type checker (mypy), that will hopefully be fixed soon. Both Pyright and Pyre support recursive type aliases right now. Open one of their playgrounds and you'll see that the following, intuitive definition

  JSON = Union[
      None,
      bool, 
      int,
      float,
      str,
      List['JSON'],
      Dict[str, 'JSON'],
  ]

works without issue.

>In fact, you admit that even fixed JSON can be difficult without a 3rd party library.

Sure, but that's just because the standard library (for now?) doesn't offer deserialization with runtime validation. Java doesn't have that either, if you want to parse JSON you'll have to either roll your own or rely on a third party library.

Don't get me wrong, the situation isn't perfect, but especially if you use the 'right' tooling, you can in my experience already get a perfectly serviceable experience, which to me is indicative that the type system itself is fine and pythonic, it's just the tooling that is lagging a bit behind at the moment.

> this is not a limitation of the type system, but of a specific type checker (mypy), that will hopefully be fixed soon

> Sure, but that's just because the standard library (for now?) doesn't offer deserialization with runtime validation.

> it's just the tooling that is lagging a bit behind at the moment

So… the standard library has exactly the problem I articulated, and you’re violently agreeing while speaking down to me.

I think that reflects poorly on you.

Your initial statement was that the type system was a poor architectural choice because it doesn't support standard python idioms. To the best of my ability I've tried to communicate that there's an important difference between the type system and the standard library, that one can make it work and that the situation is not worse than most other typed languages.

If you think that's violently agreeing and talking down to you, then there must be some communication barrier that I don't know how to overcome.

> JSON = Union[....]

This does not define the type of a valid json object. (fe None is not a valid json object, neither is '')

A decade or so ago JSON was changed so that JSON text doesn't have to be an object, it can also be a value. 'null' (or say, '5') is therefore legal JSON.

you're right: https://www.rfc-editor.org/rfc/rfc8259.txt

Reply on news.ycombinator.com