Minimalist dependency injection in Python (2021)
oddbit.dev
oddbit.dev
When an object requires other objects for its work, pass references to those other objects into its __init__ method. That’s it.
Sorry couldn't resist. The additional dependency is the biggest problem I have with OP's approach though. Now none of my classes are easily portable across projects, and this DI framework is going to spread around everywhere.
Consistently applied, all construction gets pushed to the entry-point of the program. Congratulations, you've just discovered the so-called "composition root".
Now that all construction is taking place at once, the order matters as you can't pass a dependency to its dependent until the dependency has been constructed. But it may have its own dependencies. So now there is a topological sorting problem.
Turns out computers are really good at topological sorting. So, someone made the computer do it, and we call that a dependency injection container. Tada.
The compiler generally has better attention to detail and the ability to deal with larger object graphs than the typical human.
Honestly, in practice it worked and wasn't too bad to work with, which was interesting to behold - in practice everyone talks about how circular dependencies are bad for a variety or reasons (trying to print or process data and ending up with endless loops comes to mind), but then there just was that system that was chugging along without a care in the world.
Since now, I can go to the project's root and read step by step how the application is architected and build up and which are core dependencies and which are not.
With dependency injection containers, everything is shuffled around and I'm now at the mercy of a library to tell me what I want to know.
I guess I just don't see the problem.
That solves itself naturally through normal programming. Some function takes A as argument? Well I obviously make that A first. A requires B to setup? Well obviously I make that first.
I don't need to "sort" anything, it just follows naturally from the types and constraints of the API.
I never knew this has an explicit name.
> Turns out computers are really good at topological sorting. So, someone made the computer do it, and we call that a dependency injection container. Tada.
What exactly is a "dependency injection container"? I've searched the internet for the definition, but I've only gotten more confused by all the PHP and C#. You mention topological sorting of dependencies - does that assume a hierarchical object model or can it be used with basically anything?
EDIT:
From what I've read, it appears that a Dependency Injection Container is just an object that:
1) for each dependency type, has a `GetDependency: Type -> Object` method
2) lazily creates dependencies as required by the dependents and the dependency dependencies.
Is this accurate?
* swapping out a class for an equivalent becomes an configuration concern.
* whether a type is a singleton or transient becomes a separate concern to that class's implementation, and it can be changed in the app startup config code.
There can be drawbacks too, if done badly, but I don't think that the argument that "this doesn't do very much" is entirely relevant to the pros and cons.
I wonder why Zope died out? It has everything, it's a completely sound design, and makes all components interchangeable, and it's very pretty when done properly.
It died out because it's verbose and gets hairy quickly in the real world where not everyone is on the same page, so it never ends up being used "properly".
Go for simpler and you have a higher chance of everybody getting the intent on their first reading of code.
It's going to be hard to get people to not do DI "properly" if you simply ask them to pass dependencies in (though I am sure they still could by eg. passing classes vs instances in).
1. An add_foo instance method.
2. Arguments to the `__init__` method, ideally as *args, using add_foo internally.
3. Optionally, an @instance.foo decorator which inserts the dependency as it's being defined (again, using add_foo internally).
class MyDep:
def do_foo(self):
print("fooo!")
class Fooser:
def __init__(self, foo_dooer: MyDep):
self.foo_dooer = foo_dooer
def act(self):
self.foo_dooer.do_foo()
class FakeDep(MyDep):
def do_foo(self):
print("booo!")
# With the grand unveiling:
fooser = Fooser(MyDep())
Because you don't need dependency injection in python. You can mock out any object anywhere during your tests.
Exactly! I recommend anyone who comes in contact with dependency injection to really think about this.
Mocking, and specifically mocking with Python mock module, is a quick approach to construct faked objects that simply do not fail and which log any method execution on them. Dependency injection is usually done with more complete fakes, ideally with tested fakes (fakes which pass the same sets of tests as the original, "production" implementation does).
Monkey-patching is what allows Python developers to avoid dependency injection altogether. It has its own set of risks (basically, it might affect things you are unaware of), though you can reduce those by patching exactly at the import place of your dependency (eg. patch("mymodule.utc_now") vs patch("datetime.datetime.utc_now") — if you are not doing that today, start now! :).
Still, I consider readability the main concern when programming, with testability being second (no surprise they are closely related, as tests will document behaviour precisely). With Python's monkey-patching and mocking, you are at a large risk of producing illegible gibberish, both in tests and code.
If you can at all avoid it, you should! Simple dependency injection by passing dependencies directly in (functions or classes) makes everything explicit and trivial to comprehend (and test).
Unfortunately, the biggest problem is that the world of Python libraries is completely filled with magic side effects, so you sometimes can't effectively avoid doing either of those (it would be too costly).
If you patch it ahead of time by inserting a dummy module, that dummy module might be used to initialize other modules that you weren't targeting, so undoing the patch after the test might be very tricky.
You can sort of make it work, but it seems really hacky, so for that reason I'm currently refactoring towards DI (without frameworks). But I'm open to better ideas.
__init__ is NOT the constructor in Python, the object is already created at the time when __init__ is started. Maybe this detail doesn't matter in this article, but still teaching the wrong thing.
The "constructor" is the __new__ method (which is a static method) which will create an instance of the class. Afterwards a call is made to the __init__ method on the new instance, passing any parameters specified in the __new__ method and the newly created instance (as self).
As kissgyorgy rightly points out, it's the interpreter that performs these calls.
Python is a bit more revealing about how the sausages are made.
https://docs.python.org/3.10/reference/datamodel.html#object...
https://docs.python.org/3.10/reference/datamodel.html#object...
In Java's constructors you already have access to the instance (and could leak it outside the object if you wanted to) whereas in __new__ you don't, so it doesn't seem like __new__ by itself could be called the constructor either.
__new__ doesn't call __init__, the interpreter will call the __init__ when an instance is returned from __new__, otherwise it won't even be called.
In the __new__ method docs they mention an "object constructor expression" which I take to mean:
person = Person("Bob", "Smith")
Yet oddly we have no mention of constructors or constructor expressions in:
https://docs.python.org/3/tutorial/classes.html
And the act of instantiating a new instance of a class is referred to as "“calling” a class object)"
https://docs.python.org/3/tutorial/classes.html#class-object...
Would you not agree that the docs could be a bit more consistent with regards to terminology?
If you are using something like Java or C++ then it's easy to see the benefits (some C++ developers would disagree though, read https://accu.org/journals/overload/25/140/pamudurthy_2403/).
More code = more cognitive overhead
Honestly, I don't know how you came from "my object needs objects A, B, and C to work" to the "it's cognitive overload". Your object still needs A, B and C to work, just with any other approach it's implicit which is spooky action at a distance in its finest and way more cognitive load.
def main:
a = ...
b = ...
c = ...
my_object = MyObject(a,b,c)
my_object.do_something()
def main:
my_object = make_my_object() # constructs my_object as above
my_object.do_something()
Magic, monkey-patching mess of dependency injection libraries is terrible.
Dependency injection as a concept is simple functional approach to imperative and OO programming.
> If a, b and c are MyObject's "dependencies", that's dependency injection.
In practice, though, at least coming from a Java/.NET background, DI would be understood as using something like a decorator/annotation/some registration mechanism to let the framework handle giving you a valid instance of the class that you're after, rather than straight up using the constructor whilst filling out the parameters manually.
While your example is technically true, it wouldn't be the answer that anyone in a programming interview (at least for those languages) might expect you to provide as an example.
That said, adopting the composition root approach in a Java project (or using a god-object for all of the services) was oddly liberating and felt way more simple than mucking about with appeasing Spring's @Autowired annotation and its finnicky nature.
The fact that most of that terminology is misunderstood simply supports that claim above. The fact that interviews are also generally silly is besides the point too. :)
Still, none of that should be a benchmark for what something is. Dependency injection is simply about injecting a dependency instead of using it as a side-effect. You can achieve that in many ways, and the simplest is just to pass it in.
if __name__ == "__main__":
a = os.getenv("foo")
b = get_from_somewhere()
c = "some_hardcoded_variable"
my_app(a, b, c)
My original reply was in response to "please don't make Python Java", where that flavor of spooky action DI seems to be common.
To your original question: `what do I suggest`. I point to the authors example:
di = Di()
@di.register('A')
class A:
def __init__(self, di: Di):
pass
def action(self):
print("Hi from A")
@di.register('B')
class B:
def __init__(self, di: Di):
self._di = di
self._a = None
def run(self):
self.get_a().action()
def get_a(self):
if self._a is None:
self._a = self._di.get('A')
return self._a
@di.override('A') # replace class A definition with a new one
class C:
def __init__(self, di: Di):
pass
def action(self):
print("Hi from C")
di.get('B').run() # prints "Hi from C"
di.remove('B') # removes the B dependency from the registry
How about:
class A:
def __init__(self):
pass
def action(self):
print("Hi from A")
class B:
# A is default for some reason
def __init__(self, a=A()):
self._a = a
def run(self):
self._a.action()
class C:
def __init__(self):
pass
def action(self):
print("Hi from C")
if __name__ == "__main__":
if os.getenv('ENV') == "PROD":
a = A()
b = B()
# test/non-prod/whatever version
else:
a = A()
b = B(C())
In the proposed example, I would go in, read it (assuming I know to even look for this non-idiomatic use of the language) and then read through this code to figure out what should have just been an argument passed to B.
Which leads back to my original question: what is this buying me over dunder main or handing some arguments? In terms of cost, I have to understand something and walk through the code to understand something that could have been done more simply and more explicitly without this DI pattern.
Edit: formatting
You know, like have an app_init.py and instantiate all services/dependencies manually; providing them as constructor args to other instances as needed.
Use other implementations/mocks in tests setup or dev environments.
Since python does not have interfaces (unless you write classes that to nothing, except perhaps raise errors if methods are not overwritten) this does not seem to actually do DI/IoC completely.
[1]: Together with DIP as the second principle.
<Edit>Flipped IoC (e.g. factories) and DIP (interfaces for abstraction) which I seem to had memorized with swapped meanings</edit>
You can have them in earlier versions with the typing-extensions library. [0]
> For static type checking
Protocols can be used for runtime checks as well. [1]
[0] https://pypi.org/project/typing-extensions/
[1] https://docs.python.org/3.9/library/typing.html#typing.runti...
But the "why" around these design patterns have been long lost, and people will argue to death that the patterns should always be used in every language/runtime and come up with ever more vague and circular reasons until you can't argue against it.