Comment by tony | Hacker News Reader

Comment by tony | Hacker News Reader

Painpoint with type annotations: not being able to reuse "shapes" of data, e.g. struct-like fields such as TypedDict, NamedTuple, dataclasses.dataclass, and soon *kwargs (PEP 692 [1]) via TypedDict.

Right now, there isn't a way to load up a JSON / YAML / TOML into a dictionary, upcast it via a `TypedGuard`, and pass it into a TypedDict / NamedTuple / dataclass.

dataclasses.asdict() or dataclasses.astuple() return naïve / untyped tuples and dicts. Also the factory functions will not work with TypedDict or NamedTuple, respectively, even if you duplicate the fields by hand [2].

Standard library doesn't have runtime validation (e.g. pydantic [3]). If I make a typed NamedTuple/TypedDict/dataclass with `apples: int`, nothing is raised in runtime when a string is passed.

Other issues you may run into using mypy:

- pytest fixtures are hard. It's repetitious needing to re-annotate them every test.

- Django is hard. PEP 681 [4] may not be a saving grace either [5]. Projects like django-stubs don't give you completions, it'd be a dream to see reverse relations in django models.

- Some projects out there have very odd packaging and metaprogramming that make typing and completions impossible: faker, FactoryBoy.

[1] https://peps.python.org/pep-0692/ [2] https://github.com/python/typeshed/issues/8580 [3] https://github.com/pydantic/pydantic [4] https://peps.python.org/pep-0681/ [5] https://github.com/microsoft/pyright/blob/8a1932b/specs/data...

> pytest fixtures are hard. It's repetitious needing to re-annotate them every test.

Yeah, this drives me nuts as well. But Pycharm has recently started inferring types and jump-to-declaration works, so I believe obtaining that information must be possible.

Celery is another library in the group like django/pytest in that everything about it is suuuper dynamic and *kwargs-y. It drives me up a wall that I can't readily decouple task functions from their interfaces in a typesafe way. Also I don't know how to decouple the tasks from a Celery app with a pre-defined broker uri without resorting to config files/env vars - that approach simply does not unit test well.

Yes anything that dynamic is super confusing. I honestly usually don’t bother type checking unit tests. There’s definitely some value but it’s really hard to justify on existing codebases. One thing I love about type hints (rather than compiler checks) is that you can do things like that.

The "shapes of data" thing resonates a lot especially coming from TypeScript.

The runtime validation stuff has been interesting to watch. Do you use Pydantic? It's very popular but I have a hard time getting over its willingness to cast / coerce implicitly (if I mark a field as an int, and pass in a str, I want an error -- is that weird of me?).

You can type it as StrictInt etc to stop this implicit type conversion.

https://pydantic-docs.helpmanual.io/usage/types/#strict-type...

I haven't used pydantic yet. I'm conservative when adding (non-dev) dependencies since I am maintaining library packages. For other projects, it's a possibility.

> if I mark a field as an int, and pass in a str, I want an error -- is that weird of me

That is perfectly fine. And the question is why don't TypedDict, NamedTuple, dataclass raise during construction - since if they don't - we have to play it safe and do isinstance checking since we can't trust the field's types at runtime.

I suppose the idea the `typing` module [1] is to be unobtrusive: not to be involved in runtime checks.

What I want is something that does what pydantic does in standard library. I think it's a sensible request and would be indispensable for anyone wrangling data.

[1] https://docs.python.org/3/library/typing.html

You should look into the 'dacite' library! It solved most of our load-json-into-typed-dataclass woes.

I have yet to get a moderately-complex Django project working with django-stubs. It seems to break with basic idioms like custom QuerySets, and I immediately run into new bugs every time I try it. Usually I give up and use dialled-down mypy settings.

https://pypi.org/project/jsonschema-typed-v2/

While not without caveats, but I started experimenting this, and it is quite useful.

Reply on news.ycombinator.com