Design patterns you should unlearn in Python
lihil.cc
lihil.cc
First you learn what the pattern is. Then you learn when to use it. Then you learn when not to use it.
The gap between the first and third step can be many years.
I admire your optimism!
I don't agree at all. I feel that those who criticise design patterns as solution to practical problems are completely missing the point of design patterns, and the whole reason they have the name the have: design patterns. I'll explain.
Design patterns are solutions to common design problems, but "problems" isn't the kind of problems you think it is. It's "problems" in the sense that there are requirements to be met. A state pattern is a way to implement a state machine, but you still have a state machine and a state pattern if your state classes don't handle state transitions.
More to the point, look at singletons. It's irrelevant if they are implemented with a class or a closure or a module. What makes a singleton a singleton is the fact that there is an assurance that there will be a single instance of an object. Does an implementation that allow multiple instances or doesn't return the same instance qualifies as a singleton? Obviously not.
Design patterns are recurring solutions to recurring problems. They are so recurring that they get their name and represent a high level concept. A message queue is a design pattern. An exception is a design pattern. Lazy loading is a design pattern. Retries and exponential backoffs are design patterns. Etc. Is anyone arguing that Python has none of it?
So many people trying to criticise the GoF but they don't even bother to be informed or form an educated opinion.
Design patterns aren’t solutions to common design problems. They’re after the fact descriptions of solutions for design problems. That’s the issue. That’s the beef. Everyone thought of that book as a cook book instead of a naturalists’ musings on an ecosystem, which is what they are.
Those of us who designed before people discovered that stupid book were constantly asked what the differences were between this pattern and that. And the book just isn’t thick enough and Eric Gamma was just trying to complete a thesis not write a book, so despite having at least 8 years in industry before completing his masters he cocked it up. And ruined Java in the process.
We had a contemporary of Vlissades teach a class at my last company and he crystallized all of my suspicions about the GoF book and added a whole lot more.
My advice for at least fifteen years is, if you think you want to read GoF, read Refactoring instead. If you’ve read Refactoring and still want to read GoF, read Refactoring a second time because it didn’t all sink in.
Refactoring is ten times the value of GoF for teaching you how to do this trade and how to break up architectural brambles.
No, they are not. They describe a way to design something. They are called design patterns. It is in the name, already. Consider instantiating an object. You can have a factory, or you can pass a parameter object, you can implement a builder. Perhaps you can use a flywheel? In testing there is also object mothers.
Did you noticed that each term describes a very specific and concrete approach to tackle a single use case? There is no "after the fact" anything. Claiming design patterns is something you do "after the fact" reflects a complete failure of understanding what design patterns are and how they are used.
No wonder people complain about design patterns. They are complaining about their own failure to understand what they are and how they are used. Of course people complain about things they don't understand.
> Those of us who designed before people discovered that stupid book were constantly asked what the differences were between this pattern and that.
The book didn't discovered anything. At best it contributed to standardize names used for popular design patterns.
Also, the telltale sign that people are shit talking design patterns out of sheer ignorance,and willful ignorance, is the fact that there are over a dozen reference books on design patterns. Reference authors such as Martin Fowler even wrote themselves two or three books worth of patterns. But GoF is the only one being attacked all the time? Why? Because of ignorance.
If I have to implement something similar myself in C++ however, I'll use a niche design pattern.
No, that's specious reasoning. A message queue is still a message queue if you use one provided by the OS. An exception is still an exception even if it's provided as a standard component. A state machine is still a state machine regardless of how you choose to implement your concrete implementation.
Design patterns are all about the patterns followed when designing something. That's it.
It sounds like they've not had to use it for any meaningful work, and basically described a constructor. Yeah, named parameters are great, and I miss them when I don't have them, but if you think a builder only works like this
Object.setX().setY().build()
Then it tells me that you haven't built anything meaningful. It's a way of building a state and running an operation at the end in a contained manner. If your build method doesn't run some sort of validation then it's probably a waste of time, might as well just call setters, return this and call it a day if you want to be verbose and chain commands.
What isn’t entertaining is using OpenTelemetry, which takes me right back to Java for over-engineering. Moving to OTEL from StatsD cost us about 3% CPU per core, which on 32 core machines is an entire CPU lost to telemetry. Or more accurately, an entire second CPU lost to telemetry. That is not right.
Prometheus doesn’t have these problems. And isn’t trying to fix quite as many problems I’ve never had.
> Simple: we just use the language like it was meant to be used.
> Use Default Arguments Like a Normal Human
etc
The question of when a using a given pattern is appropriate is orthogonal to that. The fact that a named pattern has been defined doesn’t imply a recommendation to use it across the board. It depends on the context and on design forces, and those change with time and circumstances. Anti-patterns are patterns as well.
It’s a pity that the idea of design patterns ended up (after the pattern language craze faded) being almost exclusively associated with the specific patterns named and described in the GoF book.
No, naming them is not the main purpose, preserving and transmitting knowledge of what they are and what they are useful for, so that people aren't fofced to rediscover solutions to the same problems over and over again. [0] Naming is obviously important for that purpose, but isn't the main goal, but a means of supporting it.
[0] If this sounds like a subset of the purpose of a reusable code library, it is, which is why in languages with sufficient abstraction facilities to allow the generic implementation of a pattern to be reusable, well documented (for the “where and when to use this” piece) code libraries replace documents that have the explanation paired with implementation recipes that one can modify to one’s particular use.
Yes, this.
It's as if design patterns represents recurring patterns followed when designing something.
Wouldn't it be silly if architects spent their time shit-talking the silliness of referring to those holes added to walls to be covered by movable panes of glass as "windows"?
Somehow we're here seeing software developers do this sort of nonsense.
That book is the closest we came to establishing a common language. I remember brushing up on the names of design patterns whenever I had an interview. Ultimately, though, it didn't yield any benefit that the industry is missing now.
Like you said, the fundamental idea behind the book was that consciously naming, cataloging, and studying design patterns would improve communication among programmers. There was also an idea that studying design patterns would give beginning programmers a richer repertoire of programming techniques faster than if they had to figure them out themselves.
Looking back with decades of hindsight, my belief is that awareness and intentional use of design patterns made no difference whatsoever. Some names stuck, and would have anyway. Others didn't, and years of status as official "design patterns" in a book widely studied across the industry couldn't make them. The younger programmers I work with who had no exposure to the GoF book, and for whom "design patterns" is something that dusty old farts used to talk about, use patterns like Flyweight, Proxy, Command, Facade, Strategy, Chain of Responsibility, Decorator, etc. without knowing or needing a name for them, and they communicate amongst themselves just as efficiently as my generation did at the height of the design pattern craze.
In the final analysis, I have never looked at the less experienced programmers around me and thought, "This situation would go faster and smoother if they had studied design patterns." The generation that learned to program after design patterns had faded as an idea learned just as quickly and communicates just as well as the generation that studied them assiduously as junior programmers like I did.
> The younger programmers I work with who had no exposure to the GoF book.....and they communicate amongst themselves just as efficiently as my generation
> The generation that learned to program after design patterns had faded as an idea learned just as quickly and communicates just as well as the generation that studied them assiduously as junior programmers like I did.
I've never read GOF so I don't know if they emphasize communication, but I have read and studied many other programming pattern books and communication is low on the list of reasons to learn them in my opinion. Their only purpose for me is to organize code in a way that has been proven to "scale" along with a codebase so that you don't end up with a plate of spaghetti.
But the ACTUAL value of the builder pattern is when you want to variadically construct an object. Create the base object, then loop or otherwise control flow over other state to optionally add stuff to the base object.
Then, additionally, the final "build" call can run validations over the set of the complete object. This is useful in cases where an intermediate state could be invalid but a subsequent update will update it to a valid state. So you dont want to validate on every update.
That’s what list comprehensions are for.
> Then, additionally, the final "build" call can run validations over the set of the complete object.
The constructor function can do that too.
kwargs = {}
if is_full_moon() and wind_direction == EAST:
kwargs["baz"] = 42
thing = Thing(foo=3.14, bar=1, **kwargs)Obviously don't reach for a builder if you don't have these use cases though.
A good example from my experience might be connecting to a Cassandra cluster it other type of database that can have extremely complex distributed settings and behaviors: timeouts, consistency levels, failure modes, retry behavior, seed connector sets.
Javaland definitely had a problem with overuse of patterns, but the patterns are legitimate tools even outside of Oop.
I haven't done much research into testing frameworks in other languages, but the spock testing framework in groovy/javaland is a serious piece of good software engineering that needs singletons and other "non hard coded/not global" approaches to work well.
Spring gets a ton of hate outside of jabs, and I get it, they tried to subsume every aspect of programming and apis especially web into their framework, but the core spring framework solved complex object graph construction in a very effective way
Oh you hate "objects" but have thousand line struct graph construction code?
It's kind of sad that groovy never took off. It offered all the good parts of java with a ton of good python, ruby, and other langs with the solid foundation of the jvm for high speed execution.
But it's effectively dead. Kind of like Cassandra is effectively dead. The tech treadmill will eventually leave you behind.
Yeah, it's sad, I like Groovy a lot. It got relegated to a second-class citizen role on Jenkins, for the most part.
> Okay, maybe you want to delay creating the object until it’s actually needed — lazy initialization. Still no need for Singleton patterns.
> Use a simple function with a closure and an internal variable to store the instance:
The given example does not actually defer instantiation, which would be clear to anyone who actually tried testing the code before publishing it (for example, by providing a definition for the class being instantiated and `print`ing a message from its `__init__`) or just understands Python well enough.
But also, using closures in this way actually is an attempt to implement the pattern. It just doesn't work very well, since... well, it trivially allows client code to end up with multiple separate instances. In fact, it actually expects you to create distinct ordinary instances in order to call the "setter" (instead of supplying new "construction" arguments).
So actually it's effectively useless, and will just complicate the client code for no reason.
Refactor to favor composition over inheritance. But if you really must inherit, single over multiple, and shallow over deep. Eventually your code will less and less need super() and it'll become pointless to use it over the more explicit mechanism, which incidentally makes everything cognitively lighter.
All the code I've written since joining has used `typing.Protocol` over ABCs, simple dependency injection (i.e., no DI framework), no inheritance anywhere, and of course extensive type annotations... and our average test coverage has gone from around 6% to around 45%.
It's honestly baffling to see how insanely over-complicated most of the Python is that I see out in the wild, especially when you consider that like 90% of the apps out there are just CRUD apps.
Classic Design Patterns: Where Are They Now - Brandon Rhodes (https://www.youtube.com/watch?v=pGq7Cr2ekVM)
Python in practice: there is more ways of doing it than in any other programming language.
Oh Python, how I love and hate you.
$ python -c 'import this' | grep way
There should be one-- and preferably only one --obvious way to do it.
Although that way may not be obvious at first unless you're Dutch.
But let me dig into just one more layer. Pay attention to the hyphens used to simulate em-dashes, and how they're spaced, versus what you might think of as "obvious" ways to use them. I'm assured that this is a deliberate joke. And of course, now that we have reasonably widespread Unicode support (even in terminals), surely using an actual emdash character is the "obvious" way. Or is it? People still have reasons for clinging to ASCII in places where it suffices. Then consider that this was written in 2004. What was your environment like at that point? How old was Unicode at that point? What other options did you have (and which ones did you have to worry about) for representing non-ASCII characters? (You can say that all those "code pages" and such were all really Unicode encodings, but how long did it take until people actually thought of them that way?) On the other hand, Python had a real `unicode` type since 2.0, released in 2001. But who do you know who used it? On yet another hand, an emdash in a terminal will typically only be one column wide (just as an 'm' character is), and barely visually distinct from U+002D HYPHEN-MINUS. (And hackers will freely identify "dash" with this character, while Unicode recognizes 25 characters as dashes: https://www.compart.com/en/unicode/category/Pd) Reasonable people can disagree on exactly when it should have become sensible to use actual emdashes, or even whether it is now. Or on whether proper typography is valuable here anyway.
from functools import lru_cache
class Whatever:
pass
@lru_cache(maxsize=1)
def get_whatever():
return Whatever()
I feel the blog post is a bunch of poorly thought through strawmen. I was browsing through the singleton example and I was wondering why would anyone use buggy code to implement something it clearly was not designed to implement.
The whole article is quite subpar. I was expecting idiomatic stuff that eliminated the need to implement something, like for example implementing singletons with modules and even getter functions, but there was none of that: just strawmen.
Really disappointing.
[0]: https://fastapi.tiangolo.com/advanced/events/#lifespan-funct...
[1]: https://asgi.readthedocs.io/en/latest/specs/lifespan.html#li...
There's also code out there which replaces the cached module object with a class instance in top-level code! This is especially used to work around the prior lack of module-level `__getattr__` (and `__dir__`), added in 3.7 (https://peps.python.org/pep-0562/). But you might still need it if for some reason you want to hook into the lower-level `__getattribute__`. And Andrew Moffat's `sh` package still does this (https://github.com/amoffat/sh/blob/develop/sh.py#L3635) even though it now only declares support for 3.8 and above. (Perhaps there was simply no clear reason to change it.)
First thing I looked up in the article was "singleton", as a sanity check, whether the article is any good. And yes, it shows module level binding as alternative, exactly what I expected, because I looked into this in the past, when someone implemented API client singleton, in a case of irrelevant early optimization of something that was never a bottleneck.
Articles like this are helpful in spreading the awareness, that one should not hold a Python like one holds a Java.
That means that as soon as you import a module, initialization happens. Ad infinitum, and you get 0.5s or more import times for libraries like sqlalchemy, requests...
The result of module import is cached (which is also what makes it valid to use a module as a singleton); you do not pay this price repeatedly. Imports can also be deferred; `import` is an ordinary statement in Python which takes effect at runtime rather than compile time.
Modules that are slow to import are usually slow because of speculatively importing a large tree of sub-modules. Otherwise it's because there's actual work being done in the top-level code, which is generally unavoidable.
(Requests is "only" around a .1s import on my 11-year-old hardware. But yes, that is still pretty big; several times as long as the Python interpreter plus the default modules imported automatically at startup.)
In fact, I've pissed off some people in interviews for holding this view. We aren't really that empirical as an industry about best practices.
...especially annoying if it's only used in one continuous chain: https://github.com/bikram990/PyScep/blob/8d80bc03368ea8dc6ea...
In my opinion the singleton pattern does however not make Python code harder to test. In Python it's actually extremely handy, because its incredibly easy to mock out a singleton in your test cases.
Or do people just not do unit testing in Python using spock-like technology? And I do use the word technology because Spock is that much better than just a bunch of test scripts
So with a Singleton what you can do at least in Java land is have your service invoke the method to get the Singleton semi-global object.
So you can mock that method invocation to return a different object. So you basically have the local object to play with in your test and won't be affecting any actual real global state
But basically the article was just saying just use global variables. Does python have a means for then intercepting the value request and data assignments for that global variable for the local scope of the testing code? Or is it hardwired like a global variable, presumably is?
In python test library, you have something called 'monkeypatch' that allows you to intercept calls to specific functions or classes and set the response yourself (I mostly use it to mock API responses tbh but it can do a lot more, an really complex operations). Monkeypatch only operate in the scope of the function or file it's written in (I think. I only remember using it in unit test functions).
Yes, if they aren't in the code, it just means the programmer has figure out and carry that around mentally when reading or writing code.
My point is that the CarBuilder is not a real type that relates to the business, but something that we had to create to encode some behaviour/rules.
Some function that validates that a dict is a valid car is much more explicit that lots of different builder classes in my opinion.
E.g.
signal = Sine(freq=440.0, amp=1.0)
.rectify()
.gain(2.0)
.center()
.clip(0.5)
That way, you get the clean chaining style and efficient, streaming computation.
Since the idea behind method chaining is a bit like UNIX pipes anyways, it would be silly (and unexpected) if commands later in the chain affected parts that came before, so you basically output a new instance each time. In my example above the signal wouldn't be a sine wave anymore after the first step, so that would make even less sense.
Editing existing instances is something I'd do for performance reasons only with no return value (lets exclude errors).
If a class seems like it needs that many parameters, it is very common that one or both of these is true:
1. It is doing too much, or
2. There are things-that-should-be-their-own-classes hiding in groups of the parameters.
#2 is kind of a subset of #1, but the "doing too much" tends to be concentrated in validating relations between parameters rather than what happens after the object is constructed.
https://www.pulumi.com/docs/iac/get-started/kubernetes/revie...
In Java (or even Go) this pattern is required to enforce type safety. In Python it seems that they ignore the typing part and just pass a bunch of dicts. Looks much cleaner, even if not entirely typesafe.
1. there are certain dependencies that it was natural to access as global state - e.g. there is typically one stdout in a program and most code just prints to it 2. a class often hard codes constructors for its internal state, e.g. if you have a Graph class it would likely have associated Edge and Node classes that it calls internally to create new edges and nodes.
the big issue with both those is testing; if you want to mock the behaviour of the print statement, or you want a graph to hold nodes that track their own creation, or whatever, you have no way of overriding the internal constructors your class uses. hence dependency injection, the idea that e.g. a graph class would be defined with a node interface and you would pass it a concrete class when constructing it (some languages have type parameters for this, but even if they do you might not think to make something as simple as the node class used internally by the graph a type parameter, and more importantly you might want to set it at run time rather than compile time).
the issue was not that ruby didn't need dependency injection the concept, it's that it didn't need the sort of dependency injection frameworks that more rigid languages like java needed, you could use ruby's built in features to do the same thing.
Right, but that's still just a generic instance of "plan ahead to use a parameter (possibly with a default value) instead of grabbing something hard-coded". I don't see a proposed fix for a class that was already designed to expect its own inner node implementation instead of an abstraction, except to rewrite it. Which is why it confused me, because the rewrite would just involve adding a parameter, so that you could pass an argument to it, and making the implementation use that parameter. I suppose in statically typed languages you might have to actually define an interface type for that parameter, whatever. None of that explained to me why the Java guys were talking about needing a "framework" to accomplish this, that apparently somehow involved a ton of XML.
> the issue was not that ruby didn't need dependency injection the concept, it's that it didn't need the sort of dependency injection frameworks that more rigid languages like java needed, you could use ruby's built in features to do the same thing.
Yes, it's much the same in Python. (And yes, I have for example used `print` as an argument to a higher-order function before.)
Conflating key/value lookups (dicts) with structured data (classes). They are both useful tools, but are for different purposes. Many python programmers (Myself many years ago included!) misused dicts when they should have been using dataclasses.
There's actually a place where the original authors subclassed dict, and dynamically generate attributes of a "data class" such that it can be used with dotted attribute access syntax or dict access syntax but the `__slots__` attribute of these classes is also generated dynamically so you don't have any auto-complete when trying the dotted attribute access. It's genuinely insane lol.
Under the hood, classes are also dict lookups, so really, this is mostly about adding type checking (unless you are also using slots).
Your comment shows a hefty amount of ignorance. Repositories wrap Entity Framework because Entity Framework's DbContext & Co are notoriously complicated to mock and stub.
Once you wrap EF stuff with a repository that implements an interface, that problem vanishes and all your code suddenly is unit testable.
But I admit my tone missed the mark. It may have been much harder to do in the past, or maybe I'm missing some nuance.
But also at this point why not just have your DbContext directly implement an interface that you can mock? Surely that must be more straightforward than adding an entire abstraction layer, that you have to extend with each new usage scenario, and without sacrificing implicit transactionality.
No, you don't. At best in-memory databases represent a test double that you can use in integration tests.
If you need to write unit tests, EF leaves you no better option than to add repositories to abstract out everything and anything involving DbContext.
public class Repository<TEntity> : IRepository<TEntity> where TEntity : class
{
public async Task Add(TEntity entity)
{
_context.Add(entity);
await _context.SaveChangesAsync();
}
}
> What happened? Well, it turns out you’re always getting the same instance, no matter what parameters you pass. Your second call to Singleton(name="Bob", age=25) didn’t create anything new — it just silently reused the original object, with its original attributes. No warning. No error. Just quietly wrong.
It's not wrong, it's right. No shit it "didn't create anything new"--it's not supposed to. It's a Singleton.