Why and how to use service injection in Node.js
vramework.io
vramework.io
My node.js life improved dramatically when I stopped using the OO paradigms, and just wrote functions in native modules.
While I've never been a huge fan of this, it's also usually the least of my problems. On the plus side it makes dependencies between modules explicit which has raised some flags occasionally.
You could make abstract classes with error-throwing methods, then extend from those for the implementation.
I think most Node people would think of that as extra boilerplate, though, since DI in Node frameworks usually exists as a side effect of other technical requirements rather than as a design goal in its own right.
How would you for example deal with having a single database pool that doesn't automatically connect on startup (when requiring a class)?
I just looked up the service lookup pattern and your right! I'll need to update the documentation accordingly.
Would you mind sharing your thoughts on why it's an anti-pattern? Multiple companies I have worked with have all come up with their own solution for the issue of swapping out different services without requiring to touch the implementation code itself.
I can give my perspective only from C# OOP perspective, which is my primary language.
Dependency injection for me means, the constructor determines the dependency, and the injector will resolve what the constructor requires.
The advantages of this are:
1. Writing unit tests, I know exactly what I need to mock, enforced by the compiler. I don't ever run tests and get an error because I forget to setup a service.
2. (In my opinion) It provides hints that a class has too many responsibilities and should be broken up. I can run a linter based on amount of constructor parameters that it is probably doing too many things.
I strongly hold the opinion that the constructor defines everything needed by a class. At my company we use a mix of DI and service lookup because 'thats how its always been done', and it really makes testing unnecessarily harder. They decided when a constructor had over 3 services, to remove all the parameters and just pass a service locator because it 'looks nicer'.
So that is my experience.
Makes a ton of sense. Services themselves are passed in other services on the constructor. The reason it isn’t done in the APIFunction is because of a uniform use / functions are called via a fixed interface.
However I think there’s a decent middle ground with typescript where it can figure out what objects are being used and then only require your tests to mock those. That way you get better typescript safety.
Thank you for the feedback!
1. The mechanism used (clearing or overriding the cache) isn’t available for ESM, where mocking is significantly more challenging.
2. That mechanism is also responsible for a lot of memory leaks when any modules are stateful.
2. That doesn’t mean they’re “right” or necessarily “better”.
3. It depends a lot on the complexity of the code under test. If it’s really simple code—and especially if the stuff you intend to mock performs well and doesn’t mutate—making it easier to mock is probably going the wrong direction.
4. For most (>50%) things, yes.
5. But also for most things (off the cuff guess: >75%), you might be better off not mocking anyway.
6. If you (like me) prefer unit tests, you’ll probably get even better results by isolating as much of your code as possible from anything you’d want to mock in the first place.
I've seen a number of occasions where a test using a mock passes one day then fails the next day because it turned out the mock was setup incorrectly, and the test was actually relying on wall clock, and it wasn't at all obvious that there was an issue because you could not statically analyze whether the mock was applied correctly.
Dependency injection was developed for languages that could not easily do that (like Java). It is just a design pattern that isnt that useful in python or js. You can decide to follow it if you think it will make your code cleaner/more readable.
It's interesting that you bring up Java, which compiles things into class files. You can technically "mix and match" things there to achieve mocking semantics by carefully constructing jars. The point of DI is to bring composability into the realm of the language. In JS, sure you can use complex machinery to get jest to mock out imports during a hidden compilation phase, but there's no contractual relationship between the mock and the API consumption deep in your unit. E.g. I could "mock" moment.js, but if the implementation switches to date-fns, then the mock is still a valid thing, but it's no longer useful/applicable. Whereas, if there is a contract that can be enforced via types/API/DI, then switching date util implementations will trigger compile errors.
Another reason passing dependencies as a parameter is great is to enforce constraints for a functional call. Say you have a database service that pools connections to your database. If you want to call some general function
loadPermissionRecords(services, id)
loadPermissionRecords({ …services, database: database.withConnectionLimit(1) }, id)
Is it simpler/cleaner than DI? Arguable
Facades are good if you're not sure if the API/semantics are set in stone.
For moment.js, one typically only cares to mock specific calls related to things like current time, so it often makes more sense to refactor towards turning currentness into an explicit input, either by mocking the hard-coded moment.js call inside a convoluted unit, or more ideally, by making the value an input argument by design in the first place.
The latter is harder to get to if you already have the code written, so mocking moment.js calls could be an acceptable alternative.
You don't need to bring the whole OOP factory manager baggage if you just want simple contracts.
Patching feels a bit messy, and if you patch out some imported module "mymod" in the implementation, it breaks when "import mymod" is changed to "from mymod import myfun", which should be an implementation detail.
You could do something tricky with the import system to substitute modules, I suppose? It seems like this should solve some of the same use-cases as DI, but I haven't seen a clear and straight-forward description of how to go about it in a clean way.
I think dependency injection is mostly a tool to allow multiple people to work on a single codebase. Rarely would you need it when you're doing stuff alone.
Might actually write a mini blog about it to see what people think.
To me the value depends a lot on circumstances like overall complexity, likelihood of change, availability of comparable test substitution mechanisms… and more than anything magic. I don’t mind direct DI—as in, passing dependencies to things. But I generally dislike magic DI where some framework passes dependencies based on some config or usage pattern. Those frameworks are often difficult to debug and more trouble than they’re worth.
Particularly since implementing DI in frameworks like express/koa is already drop dead simple - literally just have your first middleware add your services to the request context (Side note - this works just fine with TS, including nice auto-complete and type-hinting).
It's easy to write routes in express, yes. But every company & usually every project in the company usually does it differently. Oh this person used a glob tool to find all the route.js files. Oh this person has nested route.js files littered about. Oh this project is just a massive ultra-complicated routes.js.
A huge part of the win of DI & places like Java JAX-RS is that code one place looks fairly alike code another place. Dependency Injection frees us from having to know a project well, from having to be familiar with structure; we simply ask or tell, & there's a system there. I find it elegant that Express routing concerns, React concerns, and all the various configuration/providing systems you might need can be managed via IoC. Recent TypeScript example I found of these concerns, in Injex:
https://www.injex.dev/docs/plugins/express#get-post-patch-pu... https://www.injex.dev/docs/plugins/react#the-renderinjexprov...
There's a lot of push back against "magic", but really I think it's something that served Java extremely fantastically well, particularly the Dependency Injection/IoC systems. I think developers of those systems got a ton of value, and it was really understated just how noble & simple those worlds were, how clean & easy it was to walk up to any random controller & be able to discern more or less what data was going to get used, in what scopes. By not having all that stuff written in the ad-hoc, existentially open world of declarative code, and instead encoded as normative, regularized, low-overhead annotations, there was a kind of predictability & visualizability & simplicity that I think a lot of these practitioners probably sorely miss, now that they're probably mostly writing Node.js like nearly everyone else. ;)
Use DI judiciously where it provides the most value. Type, if you can type it (e.g. I vastly prefer explicit wireup in Guice Module.java files over Spring's "stringly typed" XML files. Type checking is more likely to catch classpath errors and typos. Event better, use compile time macro wireup like Scala's Macwire.
There's plenty of bad examples of ALL good & popular technologies. Don't be afraid, don't believe the terror being sold to you: give it a go, explore, see what value there is.
You can surely make it safer and more explicit by, for instance, creating a new wrapper for your Router or whatever. But by default the interface promotes casting/coercion of Request etc and just blindly asserting that you’re sure the middleware you’re expecting is attached.
In the past, I’ve worked around this by wrapping Express to make any middleware dependencies explicit on each route, and by making “middleware” actually just pure functions which return whatever state they derived from the Request. Essentially a Route just becomes a pipeline of composable functions. It worked very well, but in hindsight I found I got almost no value out of Express itself at that point.
Mark Seemann called it "poor man's DI" in the first edition of Dependency Injection in .NET and changed it to "pure DI" in the second edition, but "direct DI" might be a better term.
1. This one: “explicit is better than implicit” as a principle
2. Not this one: functional programming
“Pure DI” presumably insinuates the latter, but can get rather muddy (is dynamic scope “pure?” lololol) and I think calling it “explicit DI” might be more accessible and easier to understand.
Your code stays clean, because nothing on the consumer side needs to change if in prod/test or postgres/sqlite. Better still, your code doesn't even need awareness that prod/test environments even exist. Keeps things simpler in the long run.
- Refrain from using any Postgres features that don’t work [the same way] in SQLite (so much for your code not knowing about your test environment!)
- Maintain both in tandem and, presumably, test that they have the same behavior or… accept that they may not; in either case you have to accept that you might not know if your SQLite implementation is correct
I don’t think the alternative is having code which knows about the test environment. Instead, it’s: test with the real service or set of services you use, and limit the surface area under test.
That doesn’t mean you can’t use DI! It just means you have:
- data access layer tests that hit Postgres (and any other supported provider)
- unit tests of anything consuming the data access layer, injecting a provider which only produces results consistent with the behavior already tested in the data access layer
At that point, why inject SQLite? You already ~know the results you can expect from the injected service… just return them.
However I completely agree with what what's really important is the different types of services you run based on the implementation.
So for example when I develop my applications completely off grid, I use express + reaper to mimic S3 / cloud storage. When I deploy just swap it out with the AWS S3 service instead.
I also got inspired to use this pattern when building deepstream.io, it's a node server that can take any database/cache/message bus/etc and pretty much run with it (with a small library adaptor). In that case we basically say 'the minimum requirement for functionality is XYZ and if the service provides is, we are in. Super useful.
Management don't give a toss, as long as you deliver value (features and fixes) on time.
Especially with node where you do not have a dedicated sandbox per user request, programming needs to be request-agnostic. This is especially painful with mixing server-side and non-server-side code where you do have a dedicated 'sandbox' on the client side (aka browser).
On the other hand, in C# (and AFAIK Java too) interfaces are named and versioned, and if any of your code wants to interact with those interface implenentations, they need a reference to that interface type at compile time. In JS or Typescript, you just have to pass a type that is structurally identical (which does also not allow for versioning at the language level).
Additioanlly there being well defined object lifecycles, with pluggable/hookable lifecycle points, has also been a really interesting capability that DI systems often offer. This usually goes hand in hand with a plugin system, that allows for modification of the various steps- it's easy to add logging, perhaps to wrap some behaviors in transactional or persistence concerns, or to attach some additional context to objects, all without having to modify/wrap/compose your original providers directly. I think of DI as providing a better repository of "how we make things", and a better toolkit for working with how we make things. Which enables a host of aspect-oriented programming ideas. This is not so much from DI itself, but from the systems that usually power it. There is a management of providers & objects across time & lifecycles, which makes first class something that is usually embedded implicitly in our procedural code. Surfacing these bring-up/handling/tear-down systems makes consistent, makes visible & makes malleable a wide amount of the glue-code we have to work with.
Just ran across Injex the other week, which is a pretty decent looking Typescript DI library. It has a pretty minimal set of hooks (no object tear-down for example, but also almost never dealt with in js generally), but still is a good reference. It has hooks like "beforeCreateInstance" or "afterModuleCreation", to pick some random ones. These are enough to start modifying & altering behaviors of providers. https://www.injex.dev/docs/plugins#container-hooks
An example use is their Express plugin, which allows for a kind of Python Flask/Java JAX-RS ish-ish way of writing services/endpoints. There's some nice annotations to declare what route it wants to handle. Then the plugin system deals with registering all the routes with the express server. It's all very pat, dry, & repeatable. https://www.injex.dev/docs/plugins/express/#get-post-patch-p...
The managed repository of factories & objects has some interesting uses. In Spring, for example, there are a variety of *Aware interfaces, for watching the state of the IoC system, for seeing what objects there are, for seeing what's happening. I used this a long time ago in a EVE Online mapping application, to walk through all the things we had injected into the UI, reading out the values from the UI, and dumping them into an XML file. Then, at startup, I looked for that dump and injected all those saved values into the container. The result: the state of the UI was automatically saved at shut-down & reloaded at start up. It took some futzing around with Spring.NET but I didn't have to think about our code at all to do this: I just got spring to tell me what it had injected already, saved it to a file, and injected the file as configuration at start up. Spring made it easy.
It promises dependency scopes but I find this rarely needed and if it is a lookup table suffices.
It promises to create a graph of dependencies for the dependencies but I'd rather manually specify these rather than let a DI framework strangle my codebase.
It promises to let you mock out dependencies in unit tests but you can do so anyway as long as your dependencies are specified in the constructors.
Honestly, it was new, hip and trendy and even has its own buzzword "inversion of control", and it got to a point where you were thought to be a good developer if you were using DI, regardless of whether you actually should.
It got famous, people got excited, somehow people thought only good developers used it, it's everywhere, people are starting to regret it, and some point it'll only be used when necessary (give it a few more years).
Manually doing DI becomes interesting/challenging when not all of the injected objects are singletons (i.e., they are tied to a certain context) or when they have a lifecycle.