Somehow in my career I've never managed to develop the hatred for dependencies and upgrades that so many seem to have.
What's the big deal?
What's the big deal?
Your second thought might be 'oh, its because they let themselves use a huge amount of random gizmos!'.
Nope.. the actual root dependencies are relatively small in number, maybe 30-40 core dependencies.
The problem is, that the prevailing attitude in the ecosystem is to 'just use dependencies', for anything, like lpad. As 'I know, we'll just use the creditcard my daddy gave me'.
As long as your daddy is rich, you won't see a problem.
If your daddy isnt rich, you will notice that every 10 days as you run npm-build, it will proclaim 'hooray, there are now again 10-15 high-medium security in your 1500 dependencies'.
Combine this need to npm update continuously, with frontends happy disregard for typechecking and backwards compatibility, and you have a recipe for the plague,which we are currently living :-(
I consider 40 to be a lot, even for a huge enterprise project. I think without major refactoring half of them can be replaced with simple utility functions.
And that’s where this new fangled crap falls down: we have Django and .net apps running that are 10 years old, php and java ones 15-20 years old that all survived without much or any work. We have apps in js/typescript from 1 year ago that have bucketloads of deprecated, no longer supported, mandatory update warnings. It’s a pita so we are looking to move back to php or/and Django.
I can see that in a startup with a dedicated team, this would be no issue; you are on top of all and working daily.
Note as well that for some products (the more regulated of our products; financial/health/gov) have strict dependencies we all have to vet; in our experience of the past 4 years, it’s not a hallucination that npms change so often without any useful new features or fixes; just some useless dep updates that sometimes break things and weren’t needed, at all (you do not always need the latest version of everything ya know); I suspect (but maybe someone here knows) is that npms often update because it gives the authors more kudos somehow (as in; if posting many videos on TikTok ups your engagement; does that work for npm updates and new useless npms too?).
This happens a lot with the frontend ecosystem (where everything is a peer dependency of either react or webpack / rollup / whatever build system) but can happen on the backend too (where many things are plugins / middlewares of express or whatever server you're using and carry a peer dependency on express).
This is made more evident by the fact that, in its latest iterations, the npm project decided to install peer dependencies by default and break if their collective ranges are not compatible (before it just warned), essentially breaking CI if peer deps are broken (I'm not saying it's a bad move, before that a lot of applications essentially failed less but more obscurely).
Another issue is the fact that the Node project moves quite fastly and new versions of dependencies that you must support for security reasons may be dependant on newer Node engines. A simple dependency migration, in that case, might become a migration towards a new version of Node, that 1) in corporate environments might not be available 2) might clash with other dependencies being still dependent on earlier versions of the Node engine 3) might deprecate or remove some of the APIs you were using.
All this makes sense per se: upgrading dependencies is a complex topic that shouldn't be taken lightly. What makes less sense, imho, is the extreme customizability of a Node project: millions of available small dependencies that you can combine however you like, fractally (your deps do this as well, and their deps and so on).
The ecosystem would probably benefit from more monolithic dependencies that are less customizable. Rome was a project in this sense - a monolithic package of pre-configured, opinionated dev tooling - but now the cat is out of the bag. Less experienced developers will bikeshed if they can and npm is the perfect ecosystem to bikeshed your way to thousands of dependencies that are not upgraded regularly and after a couple of years become a nightmare to upgrade in bloc. That's why people are starting to write zero-deps code.
In short, they’re fine in moderation but each added is another liability. As such, it can be beneficial to take a judicious approach and only add them when the value they bring outweighs that liability.
You update dependencies as a full time job and it bites you because something breaks big way. But most likely you have some help to fix it and as you are on top you might need to put work to fix it but yeah still full time job.
You neglect updating dependencies and it bites you and you don’t even expect it just because you have to update something you did not expect will need it and you are now scrambling to do anything because you are messed up.
Besides I like writing my own code more than dealing with some lib shenanigans.
Ideally I wouldn’t have to worry about that.
If you skip any one of those, then your full-time job is longer about shipping features/value but just trying to get the damned thing to work.
I also tried prompting to ask if any dependencies have been eclipsed by more popular libraries, which seemed to produce results.
Might be worth experimenting to see if it can take some grunt work out of it. Or an automated report as a CI stage. Hmm.
From my experience, dependencies introduce complexity in a super-linear manner, and one should be very careful in introducing them. This is a general principle that goes beyond NPM.
That being said, I’m not against dependencies, but I do think you should think about them quite a lot. I can’t imagine not using Typescript, eslint or ts-jest as an example. Similarly I can’t imagine interacting with many Microsoft APIs without using their SDKs (even as painful as that can be at times). Or using not using React(or similar) for the frontend.
But we needed an OData client, and there were some available through NPM, but they were terrible. You could use them, but you wouldn’t have a good time. So we wrote our own, which is on NPM actually, but also a really good example of dangerous packages as we made a lot of breaking changes through the different versions as we were building it. If you had blindly used it, you would have had to put in a ton of maintenance. Something which isn’t really exclusive to the Node package environment. We have a C# application that uses RestSharp instead of the Microsoft SDKs (for God knows what reason) and it’s a few versions behind by now because they made some rather big changes that nobody wanted to spend time on. On the flip side, we’ve also used Dapper as an ORM and been very happy with that. So often it’s also a question of using dependencies where and when it makes sense.
But blindly going into them is a maintenance nightmare. The less usage packages see, and the fewer maintainers, and so on, also has a big impact on whether you can trust them or not. We update quite a few of our packages automatically after a one-two week period, and others we only update manually after vetting the code. So it’s always a resource question.
I've written Node/React web applications that break completely after a few months of not being updated, and after a year or more they need refactoring to remove deprecated/dead dependencies. And they always come with half a dozen security warnings that are impossible to get rid of.
I've seen the default Go linter aggregator have to drop half the linters because no-one updated them to use generics. It's annoying, but not so much that I want to dive in and fix someone else's linter.
I've spent hours learning the interface for ${useful library}, only to find they changed it in the latest release and they're not maintaining the old version.
And I'm certain that the massive use of dependencies will cause a huge security problem this decade. Best account I've read of this: https://medium.com/hackernoon/im-harvesting-credit-card-numb...
I use as few dependencies as possible these days.
1000s of packages with backwards-incompatible changes that seems to happen every few years in NodeJS land. Some packages have backwards-incompatible changes and you don't even know because you're not using the package yourself and suddenly the behavior of your code changes in some subtle way you'll find out about a few weeks after upgrading.
I think the problem is that NodeJS's standard library is either too barebones or nobody wants to use it, since there are libraries for all kinds of basic stuff and everyone ends up using different libraries (or different versions of the same library) for the same thing. HTTP clients, array/string manipulation and date/time handling are some notable examples.
Well... if only it was just deprecation messages... but some devs think the log is a good place for proselytism or adverts.
Classical examples of this are metrics clients, tracing clients, loggers, HTTP frameworks, routers, etc. Tools that are hard/impossible to spin out into a dedicated microservice but generic enough to warrant trying to share that code across many services.
These dependencies can have measurable impact on the user experience.
Some teams just toss the code over the wall so "it's someone else's problem." But I'd take a "every instance of my code in a production server is a production deployment" approach.
Being able to test your changes against real production traffic, knowing when you've made a change that is going to break services, owning that, being able to roll it back yourself. All of that is stuff I want in my dependency manager.
A dependency is more than something down inside a node_modules directory. It's a code path that executes alongside, or even directly handles incoming customer requests.
Trying to track down and upgrade my library/framework/w.e. when it's deployed across 100,000+ instances for 10,000+ services is... not fun.
Even less fun is trying to surgically update _only_ my code and not rebuild the entire world pulling in a huge pile of changes along with my changes.
Package management failed me when I tried to model deployments as a collection of stakeholders shipping code to production.
This isn't just an npm problem. This sucks in pretty much every package manager I've worked with. Don't get me started on apt... unpack an ar archive into a global namespace, run some Rube Goldberg scripts, and hope for the best.
The Nix paper gives me hope, but I never got around to using Nix in production before I moved on from large-scale deployments.
I've also used pretty much every package manager out there, and this may hurt some, but I've always loved the npm ecosystem the most.