Hacker News Reader

The top comment in that post shuts down the whole nonsense pretty quickly and firmly:

> If you have a super-generic function like that and type hinting enforced, you just use Any and don't care about it.

It's a stupid example, but even within the context of a `slow_add` function in a library: maybe the author originally never even thought people would pass in non-numeric values, so in the next version update instead of a hardcoded `time.sleep(0.1)` they decide to `time.sleep(a / b)`. Oops, now it crashes for users who passed in strings or tuples! If only there were a way to declare that the function is only intended to work with numeric values, instead of forcing yourself to provide backwards compatibility for users who used that function in unexpected ways that happened to work.

IMO: for Python meant to run non-interactively with any sort of uptime guarantees, type checking is a no-brainer. You're actively making a mistake if you choose to not add type checking.

As the author of that post, I'd like to point out the example was meant to be stupid.

The purpose was to show different ideologies and expectations on the same code don't work, such as strict backwards compatibilities, duck typing, and strictly following linting or type hinting rules (due to some arbitrary enforcement). Although re-reading it now I wish I'd spent more than an evening working on it, it's full of issues and not very polished.

> If you have a super-generic function like that and type hinting enforced, you just use Any and don't care about it.

Following the general stupidness of the post: they are now unable to do that because a security consultant said they have to enable and can not break RUFF rule ANN401: https://docs.astral.sh/ruff/rules/any-type/

> Following the general stupidness of the post: they are now unable to do that because a security consultant said they have to enable and can not break RUFF rule ANN401: https://docs.astral.sh/ruff/rules/any-type/

Okay, then your function which is extremely generic and needs to support 25 different use cases needs to have an insane type definition which covers all 25 use cases.

This isn't an indictment of the type system, this is an indictment of bad code. Don't write functions that support hundreds of input data types, most of which are unintended. Type systems help you avoid this, by the way.

> Don't write functions that support hundreds of input data types

But by it's nature, duck typing supports an unbounded number of input types and is what Python was built on.

You've already decided duck typing is wrong and strict type adherence is correct, which is fine, but that doesn't fit the vast history of Python code, or in fact many of the core Python libraries.

> But by it's nature, duck typing supports an unbounded number of input types and is what Python was built on.

You're trying to shove a square peg into a round hole. It's not about right or wrong. Either you want your function to operate on any type, attempt to add the two values (or perform any operation which may or may not be supported, i.e. duck typing), and throw an runtime error if it doesn't work--in which case you can leave it untyped or use `Any`--or you want stronger type safety guarantees so you can validate before runtime that nobody is calling your method with incorrect arguments, in which case you have to represent the types which you accept somehow.

If you want to have a method that's fully duck typed, you're supposed to use `Any`. That's exactly why it exists. Inventing contrived scenarios about how you can't use `Any` is missing the point. It's like complaining C doesn't work if you're not allowed to use pointers.

You're right that historically Python code was written with duck typing in mind, but now even highly flexible libraries like Pandas have type definition support. The ecosystem is way different from even 5-6 years ago, I can't think of any well-known libraries which don't have good typing support by now.

> You're trying to shove a square peg into a round hole

Yeah, that's literally the point of my Reddit post

Duck typing is great and Python’s type system has powerful support for it. You can for instance restrict a function to only objects with a frobnicate() method, without in any way constraining yourself on which implementation you accept. Type checking plus duck typing is very precise and powerful, and it helps me sleep at night.

Duck typing is great, for example to support a range of numeric inputs - say fixed-precison integers, floats, dynamic precision integers, and numpy arrays, pandas series, tensorflow/pytorch tensor. That duck typing can support functions with unbounded types, dos not mean that it is necessary, not generally sensible, for a particular function to support unbounded types.

But there was a conceivable way (maybe not in Python) to make a `slow_add` function very generic, yet only be defined over structures where any conceivable `+` operation is defined.

You just have to say the type implements Semigroup.

Yes, this would work if the arguments are lists, or integers, or strings. And it won't pass the typecheck for arguments that are not Semigroups.

It may not work with Python, but only because it's designers weren't initially interested in typechecking.

Challenge accepted.

    from dataclasses import dataclass
    from typing import Protocol, Self, TypeVar

    class Semigroup(Protocol):
        def __add__(self, other: Self) -> Self:
            ...

    T = TypeVar("T", bound=Semigroup)
    def join_stuff(first: T, *rest: T) -> T:
        accum = first
        for x in rest:
            accum += x
        return accum

    @dataclass
    class C:
        x: int

    @dataclass
    class D:
        x: int
        def __add__(self, other: Self) -> Self:
            return type(self)(self.x + other.x)

    @dataclass
    class E:
        x: int
        def __add__(self, other: Self) -> Self:
            return type(self)(self.x + other.x)

    _: type[Semigroup] = D
    _ = E

    def doit() -> None:
        print(join_stuff(1,2,3))
        print(join_stuff((1,), tuple(), (2,)))
        print(join_stuff("a", "b", "c"))
        print(join_stuff(D(1), D(2)))
        print(join_stuff(D(1), 3))
        print(D(1) + 3) # caught by mypy
        print(D(1) + E(3)) # caught by mypy
        print(join_stuff(1,2,"a")) # Not caught by mypy
        print(join_stuff(C(1), C(2))) # caught by mypy
    doit()

Now, this doesn't quite work to my satisfaction. Mypy lets you freely mix and match values of incompatible types, and I don't know how to fix that. Basically, if you directly try to add a D and an int, mypy will yell at you, but there's no way I've found to insist that the arguments to join_stuff, in addition to being Semigroups, are all of the compatible types. It looks like mypy is checking join_stuff as if Semigroup were a concrete class, so once you're inside join_stuff, the actual types of the arguments become irrelevant.

However, it will correctly tell you that it can't accept arguments that don't define addition at all, and that's better than nothing.

Pretty cool that you got this far though!

I think at this point one starts to fight against Python, which wasn't designed with this in mind. But cool nonetheless.

Thanks! My approach is to stop once it starts to hurt, and figure out what I should expect the type checker to miss. The type system and I are both getting better at it as time goes by. It’s not perfect, but it’s way better than not having it.

Stupid is okay. _Nonsense_ is not. Your example was nonsense, it was absurd. The moment I saw the first example I was like, this should be add_ints and should only take ints.

Imagine I say "the human body is dumb! Here's an example: if I stab myself, it bleeds!" Like is that stupid or absurd?

And yet, some Python users insistent on type hinting very dynamic Python code while trying to keep how dynamic it is.

One thing that post does do though is very clearly highlight the difference between Python's type system and say ... TypeScript's.

TypeScript's goal is to take a language with an unhinged duck type system that allows people to do terrible things and then allow you to codify and lock in all of those behaviours exactly as they're used.

Mypy (and since it was written by GVM and codified in the stdlib by extension Python and all other typecheckers)'s goal is to take a language with an unhinged duck type system that allows people to do terrible things and then pretend that isn't the case and enforce strict academic rules and behaviours that don't particularly care about how real people write code and interact with libraries.

If you include type hints from the very beginning than you are forced to use the very limited subset of behaviours that mypy allow you to codify and everything will be "fine".

If you try to add type hints to a mature project, you will scream with frustration as you discover how many parts of the codebase literally cannot be represented in the extremely limited type system.

At this point I'm fairly convinced that the effort one would spend trying to typecheck a python program is better spent migrating away from python into a language that has a proper type system, then using interop so you can still have the bits/people that need python be in python.

Obviously that isn't always possible but you can spend far too long trying to make python work.

Unfortunately with us being in the middle of the AI hype cycle, everyone and their dog is currently busy migrating to python.

I don't see why AI hype means more Python code.

State of the art AI models are all closed source and accessible through an API anyways. APIs that any language can easily access.

AAa for AI model development in itself, yes it's mostly Pyython, but niche.

Because everyone and their dog thinks they need to be ready to develop their own models, in python.

This is honestly a thing, at least in the startup world.

I think you are underestimating the massive amounts of Python code that is built around these things. Also, a lot of businesses are not really interested in using an API for an LLM, instead they will modify and fine-tune their own models and deploy in their own data-centers (virtual or physical), and that means even more Python code.

Sure, a system that only relies on token factory LLM APIs can be written in any language, but that is not the full width and breadth of the AI hype.

> Also, a lot of businesses are not really interested in using an API for an LLM, instead they will modify and fine-tune their own models and deploy in their own data-centers

You realize model training cost millions right? "a lot of businesses" doesn't pass sniff test here.

I'm not even counting the large swaths of data required to train. And the expensive specialists.

And then you'll have to retrain outdated models every so often.

There's a reason that AI has only a handful of players delivering SoTA models and these players are all worth $5B+.

SOTA LLM model training costs a lot, yes. But fine-tuning and training of smaller models is a lot cheaper.

I've trained useful vision-models that delivered business value for industrial applications on a MacBook overnight.

I’d be surprised if _anyone_ is migrating server code to Python because of AI.

Six month into learning to build a modern python app, with linters, type systems, tests, venvs, package managers, etc… I realized that the supposed difficulty of rust is drastically less than coming to speed and then keeping up with the python “at scale” ecosystem.

My strong suspicion is that such a story depends a great deal upon the personalities of the developers much more than any {tool chaos + type chaos} --- {new syntax + lifespan annotations} spectrum

I think you're forgetting how easy type annotation is.

I occasionally spend like 2h working on some old python code. I will spend say 15 minutes of that time adding type annotations (sometimes requires some trivial refactoring). This has an enormous ROI, the cost is so low and the benefit is so immediate.

In these cases migrating code to a proper language and figuring out interop is not on my radar, it would be insane. So having the option to get some best-effort type safety is absolutely fantastic.

I can definitely see your point, it's a useful analysis for projects under heavy development. But if you have a big Python codebase that basically just works and only sees incremental changes, adding type annotations is a great strategy.

If you do that you need to compile, which means you can't just distribute a text file with your python program. You need a build infrastructure for every python version, every architecture and every OS.

Have fun with that!

I don't understand this point at all. I've worked on Django codebases which have a huge set of typing problems... and while it's not 100% I get a lot of value out of type checking.

You annotate enough functions and you get a really good linter out of it!

If you're supposedly good at software and you spent too long trying to make python work consider the possibility that you're not good at software?

Python has flaws and big ones at that, but there's a reason it's popular. Especially with tools like pydantic and fastapi and uv (and streamlit) you can do insane things in hours what would take weeks and months before. Not to mention how good AI is at generating code in these frameworks. I especially like typing using pydantic, any method is now able to dump and load data from files and dbs and you get extremely terse validated code. Modern IDEs also make quick work of extracting value even from partially typed code. I'd suggest you just open your mind up to imperfect things and give them a shot.

> Python, AFAIK, has many features, a very permissive runtime, and perhaps (not unlike C++) only some limited subset should be used at any time to ensure that the code is manageable. Unfortunately, that subset is probably different depending on who you ask, and what you aim to do.

I'll get started on the subset of Python that I personally do not wish to use in my own codebase: meta classes, descriptors, callable objects using __call__, object.__new__(cls), names trigger the name mangling rules, self.__dict__. In my opinion, all of the above features involve too much magic and hinder code comprehension.

There's a time and a place for each of them:

* Meta classes: You're writing Pydantic or an ORM.

* Descriptors: You're writing Pydantic or an ORM.

* Callable objects: I've used these for things like making validators you initialize with their parameters in one place, then pass them around so other functions can call them. I'd probably just use closures if at all possible now.

* object.__new__: You're writing Pydantic or an ORM.

* Name mangling: I'm fine with using _foo and __bar where appropriate. Those are nice. Don't ever, ever try to de-mangle them or I'll throw a stick at you.

* self.__dict__: You're writing Pydantic or an ORM, although if you use this as shorthand for "doing things that need introspection", that's a useful skill and not deep wizardry.

Basically, you won't need those things 99.99% of the time. If you think you do, you probably don't. If you're absolutely certain you do, you might. It's still good and important to understand what they are, though. Even if you never write them yourself, at some point you're going to want to figure out why some dependency isn't working the way you expected, and you'll need to read and know what it's doing.

> Basically, you won't need those things 99.99% of the time

That's kind of my point. If you don't need a language feature 99.99% of the time perhaps it is better to cut it out from your language altogether. Well unless your language is striving to have the same reputation as C++. In Python's case here's a compromise: such features can only be used in a Python extension in C code, signifying their magic nature.

I don't need my car's airbags 99.99% of the time.

A lot of people want pydantics and orms.

I think it's fine to have those if it makes API design and better. In my mind, there's "code you should write" and there's "code only libraries should write".

I never understood why pydantic reimplemented attrs, but doing it much slower, instead of just using attrs.

You should try Go!

It should be banned by the geneva convention.

Can you share a little bit about what makes you form opinions when you are not even using the language? I think its fascinating how especially discussions about typing makes people shake their fists against a language they don't even use - and like your post make up some contrived example.

>I think that the goal there is to understand that even with the best typing tools, you will have troubles, unless you start by establishing good practices.

Like - what makes you think that python developers doesn't understand stuff about Python, when they are actively using the language as opposed to you?

Indeed, I'm not a regular Python practitioner. I had to use it from time to time because it's the language chosen by the tools I happened to use at that time, like Blender, or Django. In the former case, it wasn't very enjoyable (which says a lot about my skills in that area, or rather lack thereof), while in the latter case I found it quite likeable. So that's my background as far as python goes.

I must admit that I largely prefer static typing, which is why I got interested in that article. It's true that trying to shoehorn this feature in the Python ecosystem is an uphill battle: there's a lot of good engineering skill spent on this.

Perhaps there's a connection to make between this situation and an old theorem about incompleteness?

https://copilot.microsoft.com/shares/2LpT2HFBa3m6jYxUhk9fW

(was generated in quick mode, so you might want to double check).

As someone who has been writing python for years the worst mistake I have ever seen people make is not add type hints and not using a type checker.

Also not creating custom, expressive Pydantic types and using nested dicts in places. Nested dicts suck, you never know what you're getting, and it's well worth the time converting them to classes.

TypedDicts or data classes are both a good idea.

To try to tl;dr that rather long post:

> When you add type hints to your library's arguments, you're going to be bitten by Hyrum's Law and you are not prepared to accurately type your full universe of users

That's understandable. But they're making breaking changes, and those are just breaking change pains - it's almost exactly the same if they had instead done this:

    def slow_add(a, b):
        throw TypeError if !isinstance(a, int)
        ...

but anyone looking at that would say "well yeah, that's a breaking change, of course people are going to complain".

The only real difference here is that it's a developer-breaking change, not a runtime-breaking one, because Python does not enforce type hints at runtime. Existing code will run, but existing tools looking at the code will fail. That offers an easier workaround (just ignore it), but is otherwise just as interruptive to developers because the same code needs to change in the same ways.

---

In contrast! Libraries can very frequently add types to their return values and it's immediately useful to their users. You're restricting your output to only the values that you already output - essentially by definition, only incorrect code will fail when you do this.