Proposing a struct syntax for Python | Hacker News Reader

Proposing a struct syntax for Python

snarky.ca

44 pointsBerislavLopac50 comments

This is already handled with dataclasses

https://docs.python.org/3/library/dataclasses.html#module-da...

..and the syntax is way better, handling more members, with room to document, handling methods, and not conflicting with Mojo or stdlib

Doesn't support algebraic data types, though.

FWIW I think this article mentioning them at all is sort of weird because the article doesn't propose them either, lmao

Right, the article talks about "type Point = Point1D | Point2D | Point3D" syntax at the end... which would work just as well with dataclasses as with structs!

It already does too. Just remove the `type` bit and you have an alias for a union, which has been supported since typing was added (though the `|` syntax was only added in 3.10). And AFAIK the type checkers do handle completeness on unions.

Do you mean Python's lack of a labeled union type?

Pydantic does though.

    class A(BaseModel):
      status: Literal["success"]

    class B(BaseModel):
      status: Literal["error"]

    class C(BaseModel, ABC):
      __root__: Union[A, B] = Field(discriminator="status")

    aorb = C.parse_obj(blah…).__root__

    if aorb.status == "error":
      # aorb will type check as B.

You are a friend and a scholar.

I was looking for a solution in pydantic that is wildly relevant. Thanks, I didn’t realize that was implemented/existed.

My only objection is using ABC in your MRO. Haha, just a little bit overloaded with abstract base classes and using A, B, and C.

I got it eventually! Cool.

Ya know what, on reading that that does seem confusing. My brain had long since internalized ABC as a single token I didn't even notice.

I believe this is a good example of type narrowing :)

Yeah. Been spoiled by Rust.

It doesn't seem that far off though, it does have a union type and with pattern matching you can achieve pretty much the same thing (sure you may need a few extra classes if the types aren't distinct already but that seems like a small difference).

I don't know if type checkers can figure out if a pattern match statement is exhaustive yet though, that would be interesting.

Which Python 3.10 and static type checkers much of the benefits of algebraic data types is already available [1].

[1]: https://stackoverflow.com/questions/16258553/how-can-i-defin...

The typed equivalent of a sum type in python is a union (https://docs.python.org/3/library/typing.html#typing.Union). Works with pretty much every type you can think of, but has the usual union limitation that you can't have two variants of the same type (e.g. int | int | float).

Then again, you can just create newtype wrappers, and that's essentially the same thing as different constructors for a sum type.

You'd need to create newtype wrappers anyway in order to tell them apart, I'd assume. Otherwise it wouldn't really be an algebraic data type as much as a... union type.

Hence “equivalent”. Of your variants all have trivial and non-overlapping payloads you can just use them as-is, or erlang-style tagged tuples, in most cases it’ll do fine.

I was surprised at how dataclasses are not the first thing people are taught when it comes to "classes" in python. My two personal "you should probably learn how these work" things for beginners are pprint and dataclass. Goes a long long way to helping look at things.

It's because the traditional teaching focus is OOP and not data-driven design.

The part that blows my mind, on this, is that I see it in the "data science" community. An area that seems custom made for leaning on smaller or non-existent inheritance trees.

Dataclasses wind up cumbersome over time. They're nice when the defaults are what you want. After a handful of default overrides, I'd rather have not used dataclasses.

In the office job, I'd push on why the defaults didn't work out. In particular, I would be worried that I was specifically aiming to make future life a bit harder by not fitting with the defaults of dataclass.

To that end, I'm curious what defaults you didn't use?

This is the reasoning from the article for why its suggestion is better then dataclasses:

> - Typing is optional (there are still folks who don't want to lean into that and it simply isn't always necessary)

> - Class construction should be faster (which intuitively you would think shouldn't matter, but I have talked to folks where this is an actual concern, especially when startup time is critical)

> - Syntax typically leads to better tooling support since there's no ambiguity

> - Easier to teach than (data)classes, so can act as a stepping stone towards classes

> - Better semantics than dataclasses have by default (at least in my opinion )

> there are still folks who don't want to lean into that

Honestly why would you cater to people who want to write worse code? Should Python have a mode with implicit type coercion too because some people can't be bothered to type `int()`?

> Class construction should be faster.

It's Python. You accepted your code would be extremely slow going in. Fiddling with the syntax isn't going to make it fast.

Yeah, honestly I think if startup time is a real concern, python is simply the wrong choice. Slow startup is largely inherent in its design.

Not that it's not worth spending some effort to optimize, of course. But the lower bound is quite high even if you're very careful.

> - Typing is optional (there are still folks who don't want to lean into that and it simply isn't always necessary)

Someone who really wants to avoid type annotations at all costs can just use ": object" everywhere

I believe it should be ": Any".

Hey, you're leaning into it too much!

Who are people who are against typing in this day and age?

> Who are people who are against typing in this day and age?

People who don't find the benefits of typing are worth the overhead?

Python is, after all, a dynamically-typed language, so it is, surely, not too surprising that those who use it might include those who want a ... dynamic language ?

If it's too much to take the joys of Java are freely available.

> are worth the overhead?

What overhead, specifically?

I've found it reduces overhead. Rather than using English to describe the types, in the comments/docstrings, I can just type the terse type hint in the function definition, and leave the rest to the documentation renderer. If you don't document anyways, sure.

I use python daily and dont use typing. The reason being that I choose python for it readability. And I find typed python to be less readable than non-typed python. If I wanted a typed language, I wouldnt want python.

I'm on board with wanting an alternative to dataclasses that's faster, doesn't require types, and is less noisy (having to put @dataclass on its own line before every class is annoying when defining many small dataclasses).

But not having methods would produce APIs that feel really non-idiomatic. Essentially all Python types have methods, even immutable 'pure data' types. str has methods (though some things like `len` and regex search, which are methods in other languages, are functions in Python). int has methods (though they're rarely used). Immutable standard library classes – say, datetime or Path – have lots and lots of methods.

And inheritance… well, I could live without it but I've made good use of inheritance with dataclasses.

And as others have mentioned, the syntax is awkward.

I think my ideal solution would basically be dataclasses, but implemented in C and with more syntax sugar:

    # Instead of "@dataclass\nclass Foo:"
    struct Foo:
        # Allow untyped fields:
        x

        # Make this do what you'd expect, instead of
        # requiring "field(default_factory=list)":
        y = []

        # Of course, types are still allowed:
        z: int = 0

Methods and inheritance would be allowed as usual.

It might be interesting if these structs acted as true value types, so that e.g. just assigning an instance from one variable to another would make a copy of all the fields. That way you could mutate the object without worrying about sharing. But arguably, having both value types and reference types would overcomplicate the mental model.

How does this avoid name collisions with the built-in standard library module "struct"? Don't you think that's a pretty important consideration?

With the new parser (https://docs.python.org/3/whatsnew/3.9.html#new-parser), Python can have context-dependent keywords (like `match`).

Type creep can be a serious concern for dynamic language ecosystems. The Javascript/Typescript schism is a tragic case in point. It appears that young developers are indoctrinated into the belief that strong types are an invincible hammer and can't see the drawbacks that lack of flexibility can introduce. Clojure began with defstruct, and Rich Hickey regrets its addition to the language now. While python certainly is a kitchen sink language, and the addition of structs on top is probably sustainable, but I am of the belief that such suggestions need more counter-arguments lest we lose another mainstream dynamic language to naive good intentions.

What's unsaid here, is that such construct could be optimized way more for speed than data classes because:

- it's immutable

- doesn't have methods so no overloaded operator or dynamic attribute/index accesss

- shape is known and is set in stone

That's why Mojo has them, and why they are the bread and butter in Rust.

It's one of the step in the direction of a faster Python.

> I want syntax that will keep people from using collections.namedtuple() just for its attribute API and ignoring/regretting its index API.

The author is proposing a worse dataclass, because they don't want to ignore positional indexing on namedtuple?

Here's a long-running question I have; please tell me if I'm confused / missing something obvious.

Say I want to model in Python some collection of key-value pairs. It's a singleton -- I don't need a class or namedtuple that can be instantiated. So for example, I just want something like:

  SingletonDataContainer Config:
      root_directory: str = "/tmp"
      should_frobnicate: bool = False

So of course, it could be a dict. But I want type annotations, and dotted attribute access syntax (and pretty syntax-highlighting!). It's tempting to use a class with class attributes for this, but that seems wrong since then you end up with something that can be instantiated, but which you never actually want to instantiate.

Just put them in a module as variables. Import the module named and you have them as “dotted attributes”. Python doesn’t need all this OOP stuff like structs when we have dataclasses and attrs, modules and dicts, that can represent draconian C structs.

As well as modules being natural singletons you can also make your class an ABC to stop instantiation.

A module variable set to a SimpleNamespace in the typing package is also an option.

Thanks. I wonder what it would be like if Python had syntax for modules instead of them being tied to separate files. I would prefer being able to create this container in the same file as other code.

Say no more! Python? Rules? Unheard of.

    from importlib.machinery import ModuleSpec
    import importlib.util
    import sys

    spec = ModuleSpec("mymodule", loader=None)
    module = importlib.util.module_from_spec(spec)

    module.__dict__.update({"a": 42})

    sys.modules["mymodule"] = module

    import mymodule

    print(mymodule.a)  # -> 42

That's if you want a proper module. But modules can be any type and it's officially supported!

   import sys
   from types import SimpleNamespace
   sys.modules["mymodule"] = SimpleNamespace(b=37)

   import mymodule
   print(mymodule.b)  # -> 37

   # ^^^ This is literally just calling __getattr__ on the module object. Do with that what you will.

This will cross files as well but you need to make sure the code to create the module runs before you try importing it! glhf

Thanks! This was a great answer.

> Python? Rules? Unheard of.

OK, since it's Christmas, I'd like to be able to write

  Module MyModule
      a = 42
      b = 37

I think I can do that.

    import sys

    class DataModule:
      def __init_subclass__(cls):
        sys.modules[cls.__name__.lower()] = cls

    class MyModle(DataModule):
      a = 42
      b = 37

    import mymodule

    mymodule.a == 42

Or as a class decorator

    def datamodule(cls):
        sys.modules[cls.__name.__.lower()] = cls

    @datamodule
    class MyModule:
      a = 42

    import mymodule
    mymodule.a = 42

Could use a class, but access the value without ever instantiating it? Not sure if this fits the bill for what you want, or if the fact that it could theoretically be instantiated renders the approach non-suitable.

  class Config:
      root_directory: str = "/tmp"
      should_frobnicate: bool = False
  
  if __name__=="__main__":
      print(f"{Config.root_directory=}\n{Config.should_frobnicate=}")

Will you ever worry about thread safety for these values? If not, then as others have mentioned, a `config` module seems like what you want.

So basically you want something like how typescript does, where you can do this:

    const config = {
      rootDirectory: "/tmp",
      shouldFrobnicate: false,
    }

I'm far from a python expert, more like a TS dev who occasionally tries to write Python, but when I'm in that situation I find that writing one-off classes seems to be python's way of doing things. I'm not a super big fan of it myself, but it works:

    from dataclasses import dataclass
    
    def _create_config():
      @dataclass(kw_only=True)
      class Config:
        root_directory: str = "/tmp"
        should_frobnicate: bool = False
      return Config()
    config = _create_config()

I experimented with making a decorator that could do something like this, but never got it working with `mypy` so I gave up:

    @iife
    def config():
      @dataclass(kw_only=True)
      class Config:
        root_directory: str = "/tmp"
        should_frobnicate: bool = False
      return Config()

the `@iife` would make it so that `config` was a variable, not a `def`, but yeah the whole thing stank so I gave up

i want something like this for lots of reasons, but the headliner is structual typing.

It should be possible to get simple attribute access while not creating a new nominal type and not going outside the Python stdlib.

I'm constantly using NamedTuples when what i really want is an anonymous struct.

Oh no, not more unnecessary syntax

To add this to python I feel like you would have it has an external package and or write a pep about it and judge by how prevalent it becomes due to the fact that it says it will replace other features…

Don’t steal Mojo’s mojo

I dislike this general mode of defining an ADT by first having to create multiple separate nominal types and then combining them using another free-floating statement (e.g. 3 dataclasses and then a Union in Python). An ADT is conceptually one thought, so its definition should syntactically be able to be one statement too.

Reply on news.ycombinator.com