Using Enumerated Types in Python | Hacker News Reader

Using Enumerated Types in Python

johnlekberg.com

72 pointskaunta42 comments

Enums are best when combined with a language that does pattern matching and that can warn you if you forgot a value in your case statement. Python lacks this, so there is no spectacular benefit of using enums over just a list of strings.

Agreed on pattern matching, though mypy can help a lot here with exhaustive checks at the very least. They do work on enums and discriminated unions. They should document it:

https://github.com/python/mypy/issues/6366

https://github.com/python/mypy/issues/5818

There's a benefit in the cardinality during type checking when you use an external type checker. Strings have infinite possibilities an enum type is strictly defined.

Also I'm sure with reflection you can actually implement a simple enum pattern matching function on an enum that is exhaustive and throws a runtime error immediately if every possible enum case is not covered.

I'm not following this thread at all. What "exhaustive search" are you guys referring to? In a switch/case block? Python doesn't have that, unless I missed it while living under a rock.

I.e. if you're working with python Enums, you type-check for that Enum, and then the IDE does it all for you. If you want to "check" for a specific enum, then you cast to it and the runtime Enum implementation does that checking for you by throwing an error if you try create that Enum with an invalid string value. And if you've got a raw string, before it goes too-far into your code base, you cast it into the specific Enum type, and when serializing you serialize it to the string value. Easy-peasy.

AFAIK, they're talking about languages like Rust, or C/C++ with `-Wswitch` (also implied by `-Werror`, which will warn you if you haven't handled all enum cases in a match/switch statement.

as someone else in this thread pointed out [0], it's possible with MyPy when using if/elif/else, or if you did it with a dictionary lookup, you could write a very simple test that checks that the set of enum values is equal to the dictionary key members. I've done this before, it works well [1].

[0] https://news.ycombinator.com/item?id=23441023 [1] https://github.com/aws-cloudformation/cloudformation-cli/blo...

In some typed/compiled languages, a substantial benefit of using enums is that it provides some way to automatically determine if you've handled every eventuality. That's super not a part of normal Python as you've mentioned.

The second part of crimsonalucard1's post was referring to somehow hacking in that type of functionality into Python.

I think you could somehow do that with nested context managers - something like:

  with EnumChecker(e) as checker:
    with checker(enum_type.A):
      # do thing
    with checker(enum_type.B):
      # do thing
    ...

Where `e` is an instance of `enum_type`, and the inner context managers acting as if/case statements, then you could on any single execution of the code block (so you only need a single test case) ensure that you covered all states.

imagine an enum called bool with two values: True and False.

  switch(value: bool):
     case True:
         // do something
     case False:
         // do something else
  end

The above code is exhaustive in the sense that the logic covers all possible values of the enum type. The "exhaustive matching" we're talking about is similar to a pre runtime type check.

So imagine where someone writes this code:

  switch(value):
     case True:
         // do something
  end

The user did not handle the False case in his logic. Exhaustive pattern matching will catch this before the program runs and throw an error similar to a syntax or type check error. It will say that your switch function needs to handle all possible cases of bool. It needs to handle True AND False not just True.

The power of this basically allows you to encode safety of if else logic into a type and pass that safety throughout your program. It's pretty incredible, so much so that whenever this feature is part of a language I have stopped using if else statements almost completely.

Keep in mind the power of pattern matching extends beyond just exhaustive enums. An int is a form of enum as well:

     switch(value: int):
        case 0:
           // do something
        case >0:
           // do something else
     end

with powerful pattern matching features the compiler should immediately let you know that your switch statement did not handle the case "<0".

The article is incredibly misleading. Python Enums are way more powerful than what is showcased there.

First, rudimentary/basic enums are as simple as instance variables on a class and allow super-helpful IDE "auto-completion" that doesn't necessitate the use of magic-strings to at least "set" the enum. I've been using this ridiculously simple trick since 2.5 ten years ago. Magic-strings are bad, and even worse when you have a disparate code-base in a dynamically typed language. Example:

    class MyEnum(object):
        Pass = 1
        Fail = 2

    my_value = MyEnum. <-- at this point, the IDE gives you code-completion.

Second, real Enums added as a language-feature along with type-hinting are almost 100% on-par with statically typed languages. See:

    class MyEnum(Enum):
        Pass = 1
        Fail = 2

    def my_func(did_they_pass: MyEnum):
        if did_they_pass.Something:   # IDE / type-checker will go nuts here.
            print("Something")

Edit: Forgot the main point. Switch/case uses are only a small sub-set of the uses of Enums. So even basic/crappy enums are better than magic strings in a dynamically-typed language. Also, please don't use "list of strings" for enums in Python, there are much better ways to do that sort of thing that preclude unnecessary bugs popping up.

>real Enums added as a language-feature along with type-hinting are almost 100% on-par with statically typed languages

sorry i don't understand this. i'm not trying to be pedantic but: statically typed (and type-checked) language will fail to compile ... not just warn you. that's not a 1% difference that's the entire difference (because lots of people will miss or flat out ignore ide hints).

So, you don't use open("w"), do you ? You use the non magic open(os.O_WRONLY) of course.

Because magic strings are bad.

Python 3.8 has typing.Literal for this.

I will now - thanks for the tip! :)

And, moreover, this will inspire me to ask "How many other traditional magic numbers/strings/etc have I gotten used to that might be defined more readably?" strcmp, I'm looking at you!

I know your post was sarcastic/snarky but it was genuinely helpful

f.tell(offset, os.SEEK_CUR) over f.tell(offset, 1)

https://docs.python.org/3/library/os.html#os.SEEK_CUR

I wish the docs on f.tell at least mentioned the option

https://docs.python.org/3/tutorial/inputoutput.html#methods-...

Except it wasn't, os.O_WRONLY is for os.open() (equivalent of open syscall, takes numeric flags), not the built-in open() (equivalent of C fopen, takes "w" and similar strings).

You would have to use os.open

I would also have to pass the path as a first param. This was not meant to be correct, just teasy.

That's a fair point, and I'll be sure to keep an eye out for that type of magic-string usage going forward. Though most "magic string" usages are not nearly as official as the one you mention and usually occur in business-level code written by ordinary developers that may not be disciplined or experienced. Those spots most certainly need all the assistance they can get instead of relying on magic-strings, hence my comment of magic strings being bad.

>Second, real Enums added as a language-feature along with type-hinting are almost 100% on-par with statically typed languages

This is conclusive proof that you're not actually aware of the capabilities of enums and sum types in "statically typed languages". Where's pattern matching? Exhaustivity checking? Sum types? GADTs?

C and C++ don't have some features in that list like pattern matching and there is no question that they represent one of the most popular statically typed languages in history.

Honestly, over my nearly 20 year period of programming experience (half of which was not spent on python), I can't recall ever having encountered those terms/concepts. But on the plus side, I guess I have now and I'll be sure to read up on them. Maybe I know the concepts but haven't linked them to the terms.

Python 4... It's just going to be Haskell.

there are several benefits to using named constants over literal strings, and enums give you namespaced named constants

Not only that, but both static type-checking (with appropriate tooling) and convenient runtime membership checks.

Agreed. I’ve currently switched from Swift to Python and keep reaching for Swifts expressive enums and awesome pattern matching.

> Python lacks this, so there is no spectacular benefit of using enums over just a list of strings.

Enums represent a discrete set of constant values. They are extremely helpful to communicate to developers what to expect from a component. Lists of strings do not come near to offer the same functionality.

I disagree. A list of strings requires manual validation whether a value is actually in the list; enums have that built in. Additionally, enums (at least in Python) can have behaviour associated with each value, which can be useful in many contexts.

There are also good reasons why Python does not have switch/case statements, which are generally an anti-pattern; they work fine for a quick fix, but what they do is better solved with proper polymorphism.

It always feels a little clumsy when these class generators require the name of the identifier to which they are assigned.

  Clown = circus.Actor(“Clown”)

Why is it needed, instead of just creating an anonymous class?

  Clown = circus.Actor()

Or allowing the class factory to manipulate the current namespace?

  circus.Actor(“Clown”)
  bozo = Clown()

I feel like I am perilously close to answering my own question, but not close enough. I’m sure there’s a good reason, but I don’t know enough of the Python internals to figure it out.

> Why is it needed, instead of just creating an anonymous class?

I think the question should be: why do you want an anonymous class? If it's sufficiently ad-hoc that it doesn't warrant a name, you would typically use a dict.

> Or allowing the class factory to manipulate the current namespace?

I think that would just be considered too surprising and non-standard behaviour (other Python functions don't get to modify the scope they're called from).

Named tuples, the example I called out, are a good improvement over dicts for adhoc structs that don’t require full class definitions.

But you’re right. It’s hardly cumbersome to create classes that simply exist to contain data in a sightly more structured way than a dict.

> Named tuples, the example I called out, are a good improvement over dicts for adhoc structs that don’t require full class definitions.

Doesn't a lot of that improvement come from the fact that they have a name, though? I guess you could argue for an "UnnamedTuple" which only enforces tuple size and member names. But if you're able to specify the member names, it feels like the thing being represented is concrete enough that you should be able to come up with a name for it, and that having a name would be beneficial.

Recently I started to use enumeration classes instead of enums ( where it fits)

https://docs.microsoft.com/en-us/dotnet/architecture/microse...

If you develop in c#, I would definitely recommend trying it out

A better example can be found here actually: https://lostechies.com/jimmybogard/2008/08/12/enumeration-cl...

What's the basic idea here? That instead of ints or strings, your enum values themselves are instances of custom classes with whatever behavior you want?

If I would describe the use-case:

If you use switches with enums for functionality, this is a better way ( personal opinion). Since Enums quickly need more than 1 switch and this could wrap that behaviour

Your description is also pretty good

Have constantly struggled over how to deal with this problem. The biggest need for enums has been function arguments which can be one of a few values.

In the end, the Literal class from the typing library seems like the simplest, least wordy solution. Pycharm and mypy capture any mistakes you make which seem more than sufficient for my use cases.

Why not use Enum anyway, which works fine with MyPy? I don't mind the verbosity, it's a one-time commitment to what, 10 extra lines of code?

Issue is I'll now need two imports everytime I call this method, which becomes mildly annoying. I was doing this, but a lot of times the Literal type seems more than sufficient. It also seems to play real nice with pydantic so it's an added win!

I'm not sure what you mean by "two imports".

If you define your Enum subclass in the same file, you use one import for enum.Enu. If you eschew enums and use Literal, you use one import for typing.Literal.

Prefer using named tuples over enums in Python.

Named tuples and enums do entirely different things (loosely-speaking: the former are product types, the latter are sum types).

You also probably want dataclasses [1] instead of named tuples.

[1] https://docs.python.org/3/library/dataclasses.html

Or data bearing enums, which bring some of the best of all of those worlds, with (value declaration, not usage) exhaustiveness enforced at compile time:

https://github.com/klaviyo/matrix_enum

My colleague and I started making use of that pattern last year (before open sourcing the package) and found it very useful.

They do different things but in the wild are used in the same way. Data classes are not immutable so it would not be ideal for what I used named tuples for.

Data classes can be made immutable using ‘frozen=True’. It sucks that it’s not default though.

Can you give an example of how you use enums and named tuples in the same way? That just doesn’t seem to make a lot of sense to me, but maybe I’m missing something.

Reply on news.ycombinator.com