Every Dunder Method in Python

I have such a love/hate relationship with dunders. On the one hand, I get warm fuzzies from simplifying a module's interface with `__call__`. More than once I've seized an opportunity to rewrite some ugly conditional logic with a partial ordering by leveraging `__gt__`, `__lte__` and friends. `__setattr__` remains an interesting alternative to using decorators to execute some pre- or post-call behavior to a class method. (IIRC it can also be more efficient than wrapping a method with a decorator, since the `__setattr__` override gets precompiled into Python bytecode along with the rest of the class def.) Probably some other sleights of hand along these lines that I'm forgetting. I'm sure they were fun to write and I probably felt clever doing it.

Unfortunately in almost every case, dunder methods made it harder for me to collaborate with other people on the same codebase. And I get it. Tinkering with `__setattr__` leads to recursion errors if you're not careful, `__call__` introduces state in an unexpected place (wait, foo is a class instance? But but it behaves just like a function...). It's one of the only "stupid Python tricks" I can think of that lacks a clear analog in other programming languages, so polyglots without a strong Python background tend to hate them. I've tried to make the case on two separate occasions that dunder methods represent a more object-oriented approach to dealing with systemic complexity than type dispatch, and I stand by this. But I concede that the benefits of making the codebase nicer and more ergonomic in some places invariably requires writing class definitions that look horrendous in other places. So it's not so much that dunder methods reduce that complexity, so much as try to contain it and hopefully prevent it from spilling out into the main execution logic.

P.S. - Trey, if you're reading this, hello from a former TA!

It took me far too long to realize that “dunder” just means “double underscore” and it wasn't some academic term from grad-level CS.

Knowing that made hem less scary

This is a great guide. Awhile back, I wrote a blog post[1], "Python double-under, double-wonder," trying to dispel one common misconception I find among new Python programmers. That is, they often think that these methods are "magical" and thus shouldn't be implemented in their own classes (by "mere mortals"). That is true for a small subset of dunder methods (e.g. `__new__`), but not true for many of them (e.g. `__init__`, `__iter__`, `__call__`, `__len__`).

A better way to think about it is that the double-underscore naming convention simply means "reserved by the core Python team" -- they offer ways to hook into widely-implemented protocols in the Python runtime.

That also means you should never invent your own dunder methods or dunder attributes. This is another mistake new Python programmers -- and even some experienced Python programmers, even some Python framework writers! -- tend to make, "imitating" this naming convention to mark parts of their own code as "magic." Don't do that: the whole idea is that the core Python team reserved this namespace (`__*__`) for Python-wide protocols.

[1]: https://amontalenti.com/2013/04/11/python-double-under-doubl...

As a native Swedish speaker (and someone quite familiar with the "Bamse" [1] comic), this usage in Python always looks funny ("dunder" in Swedish means "thunder").

[1]: https://en.wikipedia.org/wiki/Bamse

The context manager ones (__enter__ and __exit__) are really useful, especially if you're coming from something like Ruby where you're used to having blocks.

Blocks are what I miss most when moving to a Python project from Ruby, and context managers aren't some sort of replacement for them, but sometimes it is nice to construct your own context manager with custom setup/teardown logic.

> Keep in mind that you're not meant to invent your own dunder methods. Sometimes you'll see third-party libraries that do invent their own dunder method, but this isn't encouraged and it can be quite confusing for users who run across such methods and assume they're "real" dunder methods.

I don't think this is good advice. Example: the dunder methods implemented and used by the Scientific Python/PyData community (__array__, __array_ufunc__, __array_func__, etc.) are so, so, so important to that ecosystem.

or just get them from the python docs: https://docs.python.org/3/reference/datamodel.html#special-m...

Completely unhinged idea, but I have always found the traditional way of doing operator overloading (that is as special methods of a class, like Python, C++ etc.) to be a bit strange.

It would be more natural to declare the algebraic structure of one or more related classes.

For example, this is something you would do to override addition:

  class C:
      ...

      def __add__(self, other: C):
          ...

The way that would feel more natural to me is something like:

  class C:
      ...

  algebraic_structure Monoid<C>:
     def +(fst: C, snd: C) -> C:
         ...

Which would also work for more complicated structure, for example:

  class Datetime:
      ...

  class Timedelta:
      ...

  algebraic_structure AffineSpace<Datetime, Timedelta>:
      def point_minus(fst: Datetime, snd: Datetime) -> Timedelta:
          ...
 
      def vector_add(fst: Timedelta, snd: Timedelta) -> Timedelta:
          ...

      def vector_inverse(vec: Timedelta) -> Timedelta:
          ...

Where you could automatically "deduce" unspecified methods.

Im going to go and say that you can make your own Tunder methods if you dont want to mess with confusing the Dunder core.

___tunder___

This is used to build micrograd and essentially extensively used in PyTorch.

https://www.youtube.com/watch?v=VMj-3S1tku0

Excellent intro video btw

A fun use of dunder hacking is to modify SomeClass.__new__ to return something other than an instance of SomeClass. Believe it or not, this is occasionally a sensible thing to do!

But it can cause problems. For example, the Mypy type checker refuses to acknowledge that this is possible: https://github.com/python/mypy/issues/1020

I still don’t understand how people claim Python is “elegant” given the number of files with four underscores in the name get sprinkled around repositories.

No Dunder-Mifflin Method?

There's no __mifflin__ method. Such a missed opportunity

If you’re reading this, please don’t submit your coworkers to __post_init__ just write normal code, nobody cares is you have some sort of crazy justification, nobody extending that shit.