A critique of package managers
gingerbill.org
gingerbill.org
Fine, now what if you need to connect to a database, or parse a PDF, or talk to a grpc backend. What a hilariously short-sighted example.
To me, this whole article just screams inexperience.
Actually his perspective is quite reasonable. Go is in the other part of the spectrum than languages encouraging "left-pad"-type of libraries, and this is a good thing.
As my psychology professor used to say. "Smart is how efficiently use your intelligence. Or don't."
So someone pretty low IQ can be smart - Forrest Gump. Or someone high IQ can be dumb occasionally - a professor so very attuned to his research topic at expense of everything else.
In other words: when someone's knowledge is disproportionately localized/siloed to their prospective subfield or domain of expertise, it does not necessitate generalization to others.
I'm certainly not saying this is the case with this particular individual, as I'm personally not familiar with their background. I'm simply stating that it's a plausible explanation for when experts in one domain make naive assertions about another domain they might not have the same experience in.
A guy designing and then implementing a programming language has a much bigger chance to put a lot of rational thinking into the tooling like dependency manager, than a typical language consumer, who can and often is easily falling into the languages emo wars.
How is ginger bill excluded from this group? No one is more invested in a language than its creator(s).
Sure, he might have given it a lot of thought, but he came up with some completely bonkers conclusions. If you don't want dependencies, DON'T IMPORT DEPENDENCIES. Don't make your dependencies extremely hard to add.
Not clear-headed about this? https://old.reddit.com/r/programming/comments/1nbkwzt/packag...
> gingerbill[S] 1 point 2 hours ago
> So a tool that enables evil is not an evil tool?
> Let's put it this way, what does a package manager specifically (not the other distinctions I make in the article) do (other than enable bad laziness and lack of proper vetting) that is actually good?
https://old.reddit.com/r/programming/comments/1nbkwzt/packag...
And to reply to your next post:
> Getting to hell quicker is not a good thing. "Emerge on the other side quickly", the other side is still hell, you haven't emerged out of it.
I'd rather use a hammer even if there is a higher chance to smack my fingers than to have to hit a nail repeatedely with my head.
Yawn.. saw it before...next, please
I'm glad you saw through me like a Superman through a lead book. Which is to say, not at all. I wasn't even thinking of Go. Where did this come from? I never mentioned Go. I don't use it or know how it does its packaging.
Are you projecting your feelings onto me as a sort of substitute for the HN gestalt? The discussion was about package managers being evil.
Now please return to the topic at hand.
Let's say you have NPM package manager. What prevents you a rational individual from saying:
{
"depedencies": {}
}So my snarky remark was about him, not about you. I think it's ok to rewind the tree up to see what is about whom. I can sincerely apologize that I have put replies to two distinct human beings, you and that other commenter, in one paragraph. Honestly, I can see that could let to confusion.
I think we can stop now..
Language designers in general terms will fall into the "more knowledgeable than the average developer"category , but let's not pretend they're anything but mere mortals like the rest of us.
NGL Ive somehow lost the thread and can't tell if we're talking about language integrated dependency managers in the abstract (in the OP), or specifically speaking about golang, odin or something else. I don't know what the emo wars are specifically in reference to but I think we jumped the shark here.
Yes dependency hell is "bad", but we have several language and package management systems today from ninja to uv that make various, obvious trade offs. Optimizing developer time, ergonomics, reproducible builds, configuration complexity are just some of the axes these pre-existing systems focus on.
If you're extremely lucky you get to pick a system that aligns with your style of work and ideals for how software should be built. If you're not, and like the rest of us, you get stuck with everyone else's poor decisions and are forced to make do. All code is legacy code given the right time horizon, so think about software with all those manual dependencies included on disk and nowhere else. How do you safely apply those required security fixes, etc. Don't be user hostile, this will just lead to our past sins like the C of old.
From a purist perspective, you can forgo all other software that you have not written in-house / or does not come with the standard library. This is the monk approach, but outside a few niche work environments that's untenable.
Odin is "successful enough" so far. Also, you know about it, so that says something.
I have technically written more Odin than Hare (one Godbolt example, arguably two if you count my explaining how to modify the example to illustrate another problem) but that just means I have more justification to say I don't like it.
I've written a lot more Scheme and I had so thoroughly forgotten writing Scheme that I had to go read the source for myself when I got email about it decades later to be sure it wasn't just a coincidence of author names.
I'm not convinced there is space for any of the "C successor" languages in the twenty-first century and in the event space is made or given for one I doubt there'll somehow be room for more. So with today's field I would bet on Zig.
And there doesn't have to be "one winner". This isn't Highlander. It is just wonderful that there is now choice in this domain beyond just the old and obvious.
I don't see how a detailed comparing of language successfullness would bring anything valuable to the point being made about the author being experienced. That seemed just a noise.
Kind of bonkers this even needs to be said, and even then it's missed/ignored.
I'd prefer instead a more balanced title like "Remember to Consider the Costs When Using Package Managers", or whatever.
Your "more balanced title" isn't even close to what I am saying. I am saying that Package Managers are just bad and should not be used. Not "remember to consider the costs". The net cost is bad for everyone, that's why I said "evil".
Yeah, but its down right stupid to do so.
The title isn't even misleading or part of a Motte-and-bailey argument.
People just hear "Package Managers are Evil" and assume that the author means you shouldn't use third party dependencies. Which is NOT what's being argued.
But I guess you'd know that, if you read passed the title.
You should try reading the article before passing judgement.
Its not like the article is called "5 facts that will make you hate package managers. Number 5 will shock you"
I a little annoyed that HackerNews post renamed it to "A critique of package managers" because that implies very different connotations. I'd view an article written like that as if I have some criticisms that could be addressed, rather than the entire concept being bad from the start.
What I'm saying is that you have failed in this argument. You hardly even attempt to make it. Thus clickbait.
You said "this is why I am saying it is evil, as it will send you to hell quicker."
Okay, so then it's up to you to prove this hell actually exists. But you don't. You just assert its existence -- "Dependency hell is a real thing which anyone who has worked on a large project has experienced." By framing it this way, you can dismiss anyone who claims to not have experienced this as not having sufficient experience. But reading the comments here, a lot of people have experienced a sort of "dependency hell" (the kind that's talked about in the wiki you link to) that is solved by package managers.
So that's why it's classed as clickbait -- you (admittedly) wrote a provocative headline that you don't even remotely back up.
FYI for the future since you're lamenting in many comments that people are misinterpreting you, this is why. Given that you don't really make an attempt to prove this dependency hell and package managers are evil, and you don't acknowledge anything good about them, it's reasonable to assume your bias is just that dependencies are evil at their core. It's actually the most charitable reading because otherwise you seem confused.
I think you're splitting hairs if you're saying that these points from the article argue against package managers but don't argue against using third party dependencies.
I similarly think you're splitting hairs if to consider "package managers are useful?" and "third party dependencies are useful?" as distinct points.
Third party dependencies absolutely are liabilities. You are liable to vet them, inspect their licenses and keep them updated while ensuring that they continue working with your existing code.
This is not something package managers help you do. Package managers like NPM make it trivial to skip these steps entirely.
What is being argued for, is a more thoughtful approach to handling third party dependencies. Or at the very least, the need for people to realise that there are costs associated with bringing third party dependencies into your codebase.
Its not splitting hairs at all. Its more of an presumption on the part of a large number of readers, that the 2 points argued conflate to "Package manager suck, because third party dependencies suck and you should write everything from scratch instead".
No it isn't.
"be careful all the time" doesn't scale. Half of all developers have below-average diligence, and that's a low bar. No-one is always vigilant, don't think that you're immune to human error.
No, you need tooling, automation to assist. It needs to be supported at the package manager side. Managing a site where many files are uploaded, and then downloaded many times is not a trivial undertaking. It comes with oversight responsibilities. If it's video you have to check for CSAM. If it's executable code, then you have to check for malware.
Package managers are not evil, but they are a tempting target and need to be secured. This can't just be an individual consumer responsibility.
I can't speak for other ecosystems, but some NuGet measures are here:
https://devblogs.microsoft.com/dotnet/building-a-safer-futur...
https://learn.microsoft.com/en-us/nuget/concepts/security-be...
I believe that there have been (a few) successful compromises of packages in NuGet, and that these have been mitigated. I don't know how intense the arms race is now.
Yes, this is the C attitude, where you provide no safety rails or poka-yokes or, indeed, package managers, and therefore you get a lot of fragile reimplementations of package managers (autoconf, anyone?). But you get to keep the satisfaction of blaming the users.
nuget is pretty good. It helps that packages tend to be substantial things, not left-pad.
Agree, this is IMHO also a better pattern. 1-liners or even 20-liners are not worth the overhead of extracting a package. Or of depending on a package.
GNU Autoconf isn't a package manager, it's more an analogue to a setup executable on MS Windows, to detect where the user wants stuff to be installed, where the user has stuff already installed and which features the user wants.
"developers, be more conscious" isn't going to fix all the issues. In general, there are not individual effort fixes to systemic issues.
NPM is also quite a wild west when it comes to publishing packages, any kid can make an account and publish 'left-pad' kind of crap.
We already have quite safe and working setup with APT and software repositories for Debian, Ubuntu etc. While it is not so easy to publish your software to Debian, you get dedicated maintainer and all kinds of requirements you have to fulfill.
But this way all the issues with trust are if not mitigated, they are minimized and for example XZ Utils hack didn't make it to production systems and it took 3 years to prepare and pull it off.
I do not think that the two are cleanly separable. They are client and server ends of the same system.
And I think my point is that I view it as more of a server (registry) and governance problem than the OP author does.
Despite the fact that my employer also has an internal package feed, the security of nuget.org and the central public feed is intrinsic to the security of the whole system.
Nuget was closer to the NPM end of the spectrum, but has tightened up considerably over time. Particularly the "Package ID Prefix Reservations" feature tells me that package names that start with certain words are owned by the relevant entity, be it "System." or "Azure." from Microsoft, or "AWS" from Amazon.
This is important as it's used to distribute SDKs and optional but standard library components and updates.
There is certainly junk on there, but not much load-bearing junk.
My argument was that this concept is not the problem.
Problem is in governance of NPM while NuGet or Maven are stricter and therefore it is registry governance problem.
But on the other hand NPM is much more popular than any other registry.
Then we're in agreement that the article's author has the wrong end of the stick, by focusing on the client end of the file transfer connection.
The popularity of such repositories and package managers are due to users of them.
And the concepts are trivially separable in my opinion. A package manager uses a repo of packages to download from. You don't need a package manager to use a repo. And a package manager could be just local to your machine and thus not need an external repo either. I know in practice the two are combined but that doesn't mean they are not distinct concepts.
For NuGet or Maven I think dependency hell is not something you run into and I don’t have package manager manager for those languages.
There should be enough trust just like I can do sudo apt install.
His take screams „I want to push my niche approach and promote my language from my Ivory Tower of language creator”. He still might not have any relevant experience building businesses line software just like O don’t have experience with building compilers or languages.
Btw the Js ecosystem also has quite a few good packages (and a ton of terrible ones, including some which everyone seems to consider as the gold standard).
Anyone I want to work with on a project is going to have to have the same frustration and want to work on the project less. Only even more because you see they downloaded version 2.7.3-2 but the version I use is 2.7.3-1.
Odin's compiler knows what a package is and will compile it into your program automatically.
`base` is intrinsically necessary to port Odin. `core` seems to be its standard library, your `libc`, `xml`, etc.
And `vendor` is everything else. So you basically get the Python's '`core` is where packages go to die' approach iff they take backwards compatibility seriously. Otherwise, they have breaking changes mid-language version change.
EDIT: Package collections not packages per gingerBill.
And we will take backwards compatibility seriously when we hit 1.0, and only "break" on major versions.
I'm talking about post 1.0 language choices:
- Choose backwards compatibility. Packages frozen in time, you get "Packages go to std to die." - Choose to break backwards compatibility. The ecosystem is split, some choose to go Odin 2 some are Odin 3.
Then those people will have to manage dependencies, which is a hell on its own. Which will cause problems. Because people are super lazy so they will automate it. In the end only thing no package manager gets you is multiple package managers to juggle.
Many languages started without package managers and eventually got them - Java, JavaScript, Python, C, C++
I know people are lazy and will automate hell. That's the entire point of the article: not everything that can be automated ought to be automated.
And the argument about multiple package managers to juggle is only the case IFF there are multiple competing ones, which with Odin, I honestly doubt it would happen if we enforced what a package is in the language. I just don't want to officially endorse one ever because I do view them to be evil.
And I don't care many languages started without them, I am not going to give in.
I don't see how changing package definition is going to help. JS had no concept of package and it was bolted on with NPM. If Odin becomes big enough, the community will override the will of the author.
Plus I don't see huge benefits to not having a package manager other than saving disk space.
Security isn't that much meaningfully better than NPM.
Trust problem exists regardless of package manager.
And people aren't far to trusting, but far too lazy. And importing packages gets job done quickly.
Dunno, hanging out in the Odin discord, it definitely attracts a crowd that thinks similarly to Bill. All the "automate everything" crowd have definitely gone to Zig, where you can create automated monstrosities with the comptime stuff and build.zig files. And the crowd that likes NPM gravitates to Rust. So Odin is just fine IMO. People on the discord share libraries that actually do things, versus an entire dependency to write a few basic procedures.
And speaking of JavaScript, nowadays ES6 does have an idea of what packages/modules/libraries are and it's so much better. All my JS dependencies for my Rails projects are just .esm.js files. I choose modules carefully, don't pull in obfuscated files, read the source, so I have 2 JS dependencies in one project and a single one in another, I write the rest myself in vanilla JS and life is great.
Sure, hence the big enough part. If you get big enough, you'll get people who are using it as a day job language, not their special darling. Having used JS and Java without package managers in a professional setting, they sucked to use.
You import a package, run the main program, see compiler/browser errors then search local repo or th Net for the missing library. Essentially you're the package manager. Which does little for bloat. You can still have a folder and import stuff en masse.
> Odin's compiler knows what a package is and will compile it into your program automatically.
...the word "automatically" should be dropped. Of course compilers compile any supplied dependency "automatically", but it is so obvious that we don't often use the adverb just for that.
They often don't though. Rust, C, C++ need either long command line invocations or a build system for anything beyond hello world. Zig needs a build file for anything beyond hello world.
With Odin, you just invoke "odin build ." and all your dependencies are taken in without needing a build system, build file, make file, etc...
So how do you explain such a system to someone? This is a genuine question I am not sure how to answer.
But yes, Odin builds it into their compiler. Rust doesn't but does have Cargo. Both are easy, as far as typical usage goes. Rust automates dependency management, Odin doesn't automate it per se but does make it easy. Which is what the whole discussion is about. A bunch of HNers whining that Odin makes it too hard, even though everyone sane uses Git anyway, and you can add dependencies using Git, and Odin will compile them without a build tool.
So for a Rust project you use Cargo + Git, for an Odin project you use Odin + Git, for a C/C++ project you use Meson (or something else if you hate life) + Git. In the end it's mostly the same, Bill just doesn't seem to want to deal with an NPM or Crates.io situation (and fair enough!).
There is no technical reason the compiler can do the job of build system, but they are typically separated because of separation of concerns. Rustc needs not be tightly coupled with Cargo---it just has to understand enough of package concepts (`--extern`) for the actual compilation, so they can be independently managed and evolve. Odin's would be the polar oppsite, while Zig's approach is somewhere in the middle (compiler-as-a-library).
Both Zig and Rust supply all of what you needed in the box, so all that Odin is doing here is commingling these features inside a single executable - there's no end user benefit that I can see.
Odin's compiler does more than rustc or clang, about the same as javac and less than Zig or Go's executables.
Some distos might try to support multiple versions of a library. That could require installing it to different prefixes instead of the default. Thus, the build system will have to comprehend that.
Literally all code I write runs on Windows, macOS, Android, and Linux. In roughly that order of priority. No I do not and will not use WSL2, it’s an abomination.
To give a concrete example, you said that javascript does not have a definition of a "package" in its langauge. But what does that really mean, and why should it lead to package manager managers? Because for me, a person who has worked with javascript just a little bit, I know package.json exists and most of the package managers I've worked with agree on what the contents of this file mean. If we limit our understanding to just npm, yarn and probably bun, we don't see how that causes or contributes to the dependency hell problem (sure it exists, but how?).
You said that Go mitigates the issue of dependency hell to some degree, but this is an interesting thought, give it more exploration! Why should something like Go not have this problem not be not as severe as in Javascript?
I may not remember the details of what you said in the article and I would like to check, but currently I can't access the site because it times-out for me.
Dependencies do suck but it is because managing a lot of complicated code sucks. You need some way to find issues over time and keep things up to date. Dependencies and package managers at least offer us a path to deal with problems. If you are managing your own dependencies, which I imagine would mean vendoring, then you aren't going to keep these dependencies up to date. You aren't going to find out about exploits in the dependencies and apply them.
No, the alternative is to imagine how many issues we would be in if every project pulled in 5 different SSL libraries. Having one that everybody uses and that is already installed on everyone's system is avoiding dependency hell. Even better if it's in stdlib.
I sympathise with the arguments but IMO laziness will always win out. If Rust didn't have Cargo to automate dependency hell, someone would create a third party script to fill the gap.
When I worked at Google every single dependency was strictly vendored (and not in the mostly useless way that Cargo vendors things). There was generally only one version of a dep in the mono repo, and if you wanted something.. you generally got to own maintaining it, and you had to make sure it worked for every "customer" -- the giant CI system made sure that you knew if an upgrade would break things. And you reached out to stakeholders to manage the process. Giant trains of dependencies were not a thing. You can do that when you have seemingly infinite budget.
But technology can indeed make it worse. I love Rust, but I'm not a fan of the loose approach in Cargo and esp Crates.io, which seems to have pulled inspiration from NPM -- which I think is more of a negative than positive example. It's way too easy to make a mess. Crates.io is largely unmoderated, and its namespace is full of abandoned or lightly maintained projects.
It's quite easy to get away with a maze of giant transitive deps w/ Cargo because Rust by default links statically, so you don't usually end up in DLL hell. But just doing cargo tree on the average large Rust project is a little depressing -- to see how many separate versions of random number generators, SHA256, MD5, etc libs you end up with in a single linkage. It may not be the case that every single one is contributing to your binary size... but it's also kind of hard to know.
Understanding the blast radius of potential issues that come from unmoderated 3rd-party deps is I think something that many engineers have to learn the hard way. When they deal with a security vulnerability, or a fundamental incompatibility issue, or have to deal with build time and binary size explosions.
I wish there was a far more mature approach to this in our industry. The trend seems to be going in the opposite direction.
I feel we need more of these kinds of distros so you don't need to manage dependencies directly from upstream and deal with the integration effort yourself. What if we had a Rust disto following this same model, where there is only one version of each dep, some reasonable curation, and also you had nice clear release cycles? I feel that could real boon for the industry.
I dunno maybe what is needed is a crates.io alternative that is highly highly moderated and highly highly selective. A subscription service with a paid staff that manages the packages and makes sure their deps are minimal, consistent with each other, secure, etc.
I can see that being a service that some corporations might pay for. I just came off a gig at a medical devices company that was using Rust and the software BoM side of things kept me up at night. The list of dependencies in the root workspace was long, and in my imagination, full of terrors.
Maybe they'll be fine, but it's not a practice I would recommend if I were starting such a project from scratch.
But it’s annoying to have to deal with 3 different time libraries and 3 different error creation libraries and 2 regex libraries somehow in my dependency tree. Plus many packages named stuff like “anyhow” or “nom” or other nonsense words where you need to google for a while to figure out what a package is supposed to do. Makes auditing more difficult than if your library is named structured-errors or parser-combinator.
I don’t like go programming language but I do like go tooling & go ecosystem. I wish there was a Rust with Go Principles. Swift is kinda in the right ballpark, packages are typically named stuff that makes sense and Swift is closer to Rust perf and Rust safety than Go perf and Go safety. But Swift is a tiny ecosystem outside of stuff that depends on the Apple proprietary universe, and the actual APIs in packages can be very magical/clever. ¯\_(ツ)_/¯
Python is much older than Go, and has had more packages move from 3rd party into the stdlib to become a "battery", and then atrophy over the years while people move back to 3rd party alternatives with more features that are actually receiving maintenance. Eventually some of those modules were removed from core.
Perhaps the Go model only works when you have a very dedicated core group (for Go, mostly Google employees) around to continuously build and maintain the Cathedral of the standard library + toolchain together. Golang feels very much like UNIX (eg FreeBSD) for this reason, and Rust/Python more like Linux.
I mostly write C++ whose committee is incompetent and sniffs glue. And I deal a lot with Khronos committed who design pure garbage. “Design by Committee” is a pejorative for a reason.
Unless I am reading this (https://index.ros.org/?search_repos=true) wrong.
Something like OpenCV in the language stdlib is my idea of "batteries included". Like I said: what you consider batteries are not what people in my line of work consider batteries; Odin advertises "batteries included" but looking through the list I wouldn't use any of that in my day to day.
That's why Rust has a small stdlib of just the essentials, because doing so keeps things general, and everyone gets to choose their own idea of batteries.
Possibly but not guaranteed. Some other languages without a built in package manager haven't had an external one manage to take over the ecosystem, most (in)famously C and C++, while others have.
I rather appreciate that C and C++ don't have a default package manager that took over - yes, integrating libraries is a bit more difficult, but we also have a lot of small, self-contained libraries that just "do the thing" without pulling in a library that does colored text for logging, which pulls in tokio, which pulls in mio, which pulls in wasi, which pulls in serde, which is insane.
RPM and APT packages are also usually not maintained by the upstream developer but by distro developers who care about making different packages work together so you don't get the dependency hell problem as a user.
Slackware Linux does precisely that.
I'm a Slackware user. Slackware does have a package manager that can install or remove packages, and even a frontend that can use repositories (slackpkg), but it does have manual dependency resolution. Sure, there are 3rd-party managers that can add dependency resolution, but they do not come with the distro as default.
This is a very personal opinion, but manual dependency management is a feature. Back in the day, I remember installing Mandrake Linux 9.2 and activating the (then new-ish) framebuffer console. The distro folks had no better idea than to force a background "9.2" image on framebuffer consoles, which I hated. I finally found the package responsible for that. Removing it with urpmi, however, meant removing all the graphical desktop components (including X11) because that stupid package was listed as a dependency of everything graphical.
That prompted me to seek alternatives to Mandrake and ended up using Slackware. Its simplicity had the added bonus of offering manual dependency resolution.
[0] https://en.wikipedia.org/wiki/Dependency_hell
I find it strange that they use a term with a common meaning, link to that meaning, and then talk about something else?
The second thing is that their version of dependency hell - having lots of dependencies introducing lots of bugs that you would not have written - is not my experience. 99% of the time, my bugs are in my own code, lol. Maybe once you become a much better programmer than me, you stop writing bugs in your own code and instead start having to deal with bugs in the PNG parsing library you depend on or something, and at that point writing your own PNG parsing library becomes a good use of your time. But I'm certainly not at that point.
I've had to fix bugs in dependencies of course. Here is one I fixed yesterday [0]. But it's much closer to the exception than the rule.
1) Have problem that feels too complicated to hand-code.
2) Go on Internet/forums, find a library. The library is usually a small, flat collection of atomic functions.
3) A senior engineer vets the library and approves it for use.
4) Download the stable version: header file, and the lib file for our platform (on rare occasions, build it from source)
5) Place the .h file in the header path, and the lib file in the lib path; update the Makefile.
6) #include the header and call functions.
7) Update deployment scripts (bash script) to scp the lib file to target environment, or in some cases, use static linking.
8) Subscribe to a mailing list and very occasionally receive news of a breaking change that requires a rebuild.
This may sound like a lot of work, but somehow, it was a lot less stressful than dealing with NPM and node_modules today.
I find that it's the hell of transitive dependencies--you as a developer can reasonably vet a single layer of 10-30 standalone libraries. But if those libraries depend on other libraries, etc, then it balloons into hundreds or thousands of dependencies, and then you're sunk.
For what it's worth, I don't think much of this is essential complexity. Often a library is complicated because it supports 10 different ways of using it, but when you use the library, you're only using 1 of those ways. If everyone is only using 10% of thousands of transitive dependencies, the overall effect is incredibly complicated, but could have been achieved with 10-100% more short-term effort. Sure, "it took twice as long to develop but at least we don't have 10x the dependencies" is a hard sell to management (and often to ourselves), but that's because we usually choose to ignore the costs of depending on software we don't understand and don't control. We think that we're cleverly avoiding having to maintain and secure those libraries we outsourced, but most open-source developers aren't doing a great job of that anyway.
Often it really is easier to develop something from scratch, rather than learn and integrate a library. Not always though, of course.
The author asserts that most open-source projects don't hit the quality standards so that their libraries can be just included, and they'll do what they say.
I assert that this is because there's no serious product effort behind most libraries (as in no dedicated QA/test/release cycle), no large commercial products use it (or if they do, either they do it in a very limited fashion, or just fork it).
Hobbyists do QA as long as it interests them/fits their usecase, but only the big vendors do bulletproof releases (which in the desktop realm seems to be only MS/Apple)
This might have to do with the domain the author chose - desktop development has unfortunately had the life sucked out of it with every dev either being a fullstack/cloud/ML/mobile dev, its mindshare and the resources going toward it have plummeted.
(I also have a sneaking suspicion the author might've encountered those bugs on desktop Linux, which, despite all the cheerleading (and policing negative opinions), is as much as a buggy mess as ever.
In my experience, it's quite likely to run into a bug that nobody has written about on the internet ever.
I have an article on my unstructured thoughts on the problems of OSS/FOSS which goes into more depth about this: https://www.gingerbill.org/article/2025/04/22/unstructured-t...
I find myself nodding along to many of the technical and organizational arguments. But I get lost in the licensing discussion.
If it is a cultural problem that people insist on giving things away for free (and receiving them for free), then viral licenses can be very helpful, not fundamentally pernicious.
Outside of the megaprojects, my mental model for GPL is similar to proprietary enterprise software with free individual licenses. The developer gets the benefits of open projects: eyeballs, contributors, adoption, reputational/professional benefits, doing a good deed (if that motivates them) while avoiding permissively giving everything away. The idea that it's problematic that you can't build a business model on their software is akin to the "forced charity" mindset—"why did you make something that I can't use for free?"
If you see a GPL'd bit of code that you really want to use in your business, email the developers with an offer of $X,000 for a perpetual commercial license and a $Y,000/yr support contract. Most are not so ideologically pure to refuse. It's a win-win-win: your business gets the software, the developers don't feel exploited, noncommercial downstream users can enjoy the fruits of open software, and everybody's contributed to a healthier attitude on open source.
This the wrong thing to automate. You can do this manually, however it doesn’t stop you getting into hell, rather just slow you down, as you can put yourself into hell (in fact everyone puts themselves into hell voluntarily). The point is it makes you think how you get there, so if you have to download manually, you will start thinking “maybe I don’t want this” or “maybe I can do this instead”. And when you need to update packages, being manual forces you to be very careful."
I sympathise with this, but I have to respond that we have to live within existing ecosystems. Getting rid of npm and doing things manually won't make building SPAs have fewer dependencies, build would be incredibly slow and painful.
It will at least massively help prevent things from breaking unexpectedly.
It won't prevent you from having to cascade a necessary upgrade (such as a security fix) across the entire project until resolution/new equilibrium is achieved.
My solution to the latter is simply to try to depend on as few things as possible. But eventually, the cancer will overtake the project if it keeps growing.
Source: Have worked on a million-LOC Ruby app
The solution is just to depend on less and manage them manually.
The other thing is your package manager cannot go out to the internet randomly. You need it to download from a place you are comfortable with (which might or might not be the default) existing as long as you need packages, and that will keep the versions of packages you want around. If you are a company project that means an internal server/mirror because otherwise something you depend on will disappear in the future. (most of they decide nobody is using it, delete it, but sometimes it is discovered the thing is an illegal copyright violation - but you have ask your lawyers what this means for you - perhaps a license is easy to get)
You don't think making adding dependencies incredibly slow and painful would make people have fewer of them?
Same number of lines but in fewer dependencies.
But in the context of newer ecosystems or ones that are more tightly controlled things might be different. For example if apple massively expanded the swift standard library and made dependency management painful, iOS apps might end up having fewer dependencies.
I remember installing software in the early 90s: download the source code, read the README, find and download the dependencies, read their READMEs, repeat a few times. Sometimes one dependency could not compile because of any incompatibility or bug. Some could be fixed, some couldn't. Often everything ended up with a successful compilation and install and in one day of work I could have what I'm getting in a few minutes now.
Actually those were small programs by today standards. My take is that we would achieve less if we have to use less dependencies.
By the way, the last time I compiled something from source was yesterday. It was openvpn3 on Debian 13, which is still unsupported. TLDR, it works but the apt-get are a little different from the ones in BUILD.md
Part of my reproducing the build was to conduct all the library downloading in a miniconda environment, so at the end I had a reproducible recipe.
Is the original author seriously claiming that anybody was better off with the original, "pure" ad-hoc approach?
Honestly, I don't think this is true in the slightest. Rather, I hypothesize that people want to use such tooling and think the alternatives are slower, which I don't think is true.
If people actually did use fewer dependencies, people would have actually have websites that didn't take ages to load and were responsive.
So the existing ecosystems are just bad.
Regardless of how they define these terms, producing a list of hashes which function as a commitment to specific versions of dependencies is a technique essential to modern software development. Whatever the tools are called, and whatever they do, they need to spit out a list of hashes that can be checked into version control.
You could just use git submodules, but in practice there are better user experiences provided by language package managers (`go mod` works great).
A good amount of this ranting can probably be attributed to projects and communities that aren't even playing the list of hashes game. They are resolving or upgrading dependencies in CI or at runtime or something crazy like that.
Also, use git subtrees, not git submodules. What people think submodules are, are actually subtrees and most people don't know about them.
As for "good" package managers, they are still bad because of what I said in the article.
And honestly speaking: It is plain stupid.
We can all agree that abusing package management with ~10000 of micro packages everywhere like npm/python/ruby does is completely unproductive and brings its own considerable maintenance burden and complexity.
But ignoring the dependency resolution problem entirely by saying "You do not need dependencies" is even dumber.
Not every person is working in an environment where shipping a giant blob executable built out of vendored static dependencies is even possible. This is a privilege of the Gamedev industry has and the author forgets a bit too easily it is domain specific.
Some of us works in environment where the final product is an agglomerate of >100 of components developed by >20 teams around the world. Versioned over ~50 git repositories. Often mixed with some proprietary libraries provided by third-party providers. Gluing, assembling and testing all of that is far beyond the "LOL, just stick to the SDL" mindset proposed here.
Some of us are developing libraries/frameworks that are used embedded in >50 products with other libraries with a hell of multiples combinations of compilers / ABI / platforms. This is not something you want to test nor support without automation.
Some of us have to maintain cathedrals that are constructed over decades of domain specific knowhow (Scientific simulators, solvers, Petrol prospection tools, financial frameworks, ... ) in multiple languages (Fortran, C, C++, Python, Lua, ...) that can not just be re-written in few weeks because "I tell you: dependencies sucks, Bro"
Managing all of that manually is just insane. And generally finishes with an home-made half-baked bunch of scripts that try to badly mimic the behavior of a proper package manager.
So no, there is no replacement for a proper package manager: Instead of hating the tool, just learn to use it.
Package manager are tools, and like every tool, they should be used Wisely and not as a Maslow's Hammer.
> Some of us works in environment where the final product is an agglomerate of >100 of components developed by >20 teams around the world. Versioned over ~50 git repositories. Often mixed with some proprietary libraries provided by third-party providers. Gluing, assembling and testing all of that is far beyond the "LOL, just stick to the SDL" mindset proposed here.
Does this somehow prevent you from vendoring everything?
Would you also try to build all of them on every CI run?
What about the non-source dependencies, check the binaries into git?
Yes. Because in these environment soon or later you will be shipping libraries and not executable.
Shipping libraries means that your software will need to be integrated in other stacks where you do not control the full dependency tree nor the versions there.
Vendoring dependencies in this situation is the guarantee that you will make the life of your customer miserable by throwing the diamond dependency problem right in their face.
In the game development sphere, there's plenty of giant middleware packages for audio playback, physics engines, renderers, and other problems that are 1000x more complex and more useful than any given npm package, and yet I somehow don't have to "manage a dependency tree" and "resolve peer dependency conflicts" when using them.
And you just don't know what you are talking about.
If I am providing (lets say) a library that provides some high level features for a car ADAS system on top of a CAN network with a proprietary library as driver and interface.
This is not up to me to fix or choose the library and the driver version that the customer will use. He will choose the certified version he will ship, he will test my software on it and integrate it.
Vendoring dependency for anything which is not a final product (product as executable) is plain stupid.
It is a guarantee of pain and ABI madness for anybody having to deal with the integration of your blob later on.
If you want to vendor, do vendor, but stick to executables with well-defined IPC systems.
If you're writing an ADAS system, and you have a "dependency tree" that needs to be "resolved" by a package manager, you should be fired immediately.
Any software that has lives riding on it, if it has dependencies, must be certified against a specific version of them, that should 100% of the time, without exceptions, must be vendored with the software.
> It is a guarantee of pain and ABI madness for anybody having to deal with the integration of your blob later on.
The exact opposite. Vendoring is the ONLY way to prevent the ABI madness of "v1.3.1 of libfoo exports libfoo_a but not libfoo_b, and v1.3.2 exports libfoo_b but not libfoo_c, and in 1.3.2 libfoo_b takes in a pointer to a struct that has a different layout."
If you MUST have libfoo (which you don't), you link your version of libfoo into your blob and you never expose any libfoo symbols in your library's blob.
The vendoring step happens at something like Yocto or equivalent and that's what ends up being certified, not random library repos.
And in addition: Yocto (or equivalent) will also be the one providing you the traceability required to guarantee that what you ship is currently what you certified and not some random garbage compiled in a laptop user directory.
It used to have a really bad design flaw. Example: - building package X explicitly depends on A to be in the sysroot - building package Y explicitly depends on B in the sysroot, but implicitly will use A if present (thanks autoconf!)
In such a situation, building X before Y will result in Y effectively using A&B — perhaps enabling unintended features. Building Y then X would produce a different Y.
Coupled with the parallel build environment, it’s a recipe for highly non deterministic binaries — without even considering reproducibility.
When you want reproducibility, you need to specify what you want, not let the computer guess. Why can't you use Y/configure --without-A ? In the extreme case you can also version config.status.
Things using autotools evolved to be “manual user friendly” in the sense that application features are automatically enabled based on auto detected libraries.
But for automated builds, all those smarts get in the way when the build environment is subject to variation.
In theory, the Yocto recipe will fully specify the application configuration regardless of how the environment varies…
Of course, in theory the most Byzantine build process will always function correctly too!
It's better than before but you still need to sandbox manually if you want good reproducibility.
Honestly, for reproducibility alone. There is better than Yocto nowadays. It is hard to beat Nix at this game. Even Bazel based build flows are somewhat better.
But in the embedded world, Yocto is pretty widespread and almost the de-facto norm for Linux embedded.
You're providing a library. That library has dependencies (although it shouldn't). You've written that library to work against a specific version of those dependencies. Vendoring these dependencies means shipping them with your library, and not relying on your user, or even worse, their package manager to provide said dependencies.
I don't know what industry you work in, who the regulatory body that certifies your code is, or what their procedures are, but if they're not certifying the "random library repos" that are part of your code, I pray I never have to interact with your code.
You illustrate perfectly the attitude problem of the average "gamedev" here.
You do not know shit about the realities and the development practice of an entire domain (here the safety critical domain).
But still you brag confidently about how 'My dev practices are better' and affirm without any shame that everybody else in this field that disagree is an idiot.
Just to let you know: In the safety critical field, the responsibility of the final certification is on the integrator. That is why we do not want intermediate dependency to randomly vendor and bundle crap we do not have control of.
Additionally, it is often that the entire dependency tree (including proprietary third party components like AUTOSAR) are shipped as source available and compiled / assemblied from sources during the integration.
Thats why the usage of package manager like Yocto (or equivalent) is widespread in the domain: It allows to precisely track and version what is used an how for analysis and traceability back to the requirements.
Additionally again, when the usage of binary dependencies is the only solution available (like for Neutrino QNX and its associated compilers). Any serious certification organism (like the TUV) will mandate to have the exact checksum of each certified binary that you use in your application and a process to track them back to the certification document.
This is not something you do by dumping random fu**ng blob in a git repository like you are proposing. You generally do that, again, by using a proper set of processes and generally a package manager like Yocto or similar.
Finally, your comment on "v1.3.1 of libfoo" is completely moronic. You seem to have no idea of the consequence of duplicated symbols in multiples static libraries with vendored dependencies you do not control nor the consequences it can have on functional safety.
I dabbled my fingers in enough of them to tame my hubris a bit and learn that various fields have specific needs that end up represented in their processes (and this includes gamedev as well). Highly recommended before commenting any further.
> So let's handle the hell manually to feel the pain better
This is far from my position. Literally the entire point is to make it clearer you are heading to dependency hell, rather than feel the pain better whilst you are there.
I am not against dependencies but you should know the costs of them and the alternatives. Package managers hide the complexity, costs, trade-offs, and alternative approaches, thus making it easier to slip into dependency hell.
You are against the usage of a tool and you propose no alternative.
Handling the dependency by vendoring them manually, like you propose in your blog, is not an alternative.
This is an over simplification of the problem (and the problem is complex) that can be applied only to your specific usage and domain.
Again, what is wrong with saying you should know the costs of the dependencies you include AND the alternative approaches of not using the dependencies?—e.g. using the standard library, writing it yourself, using another dependency already that might fit, etc.
That said I think the final takeaway is that systems that allow you to pin versions, vendor all those dependencies and resolve/reproduce the same file tree regardless of who's machine it's on (let's assume matching architectures for simplicity here) is the goal.
Note that removing 'manually' here, this still works:
> Copying and vendoring each package {manually}, and fixing the specific versions down is the most practical approach to keeping a code-base stable, reliable, and maintainable.
The article's emphasis on the manual aspect of management of dependencies is a bit of loss, as I don't particularly believe it _has to be manual_ in the sense of manually copying files from their origin into your file tree; that certainly is a real world option, but few (myself included) would take that monk-like path again. I left this exact situation in C land and would not consider going back unless adopting something like ninja.
What the OP is actually describing is a "good" package manager feature set and many (sadly not most/all) do support this exact feature set today
PS I did chuckle when they defined evil in terms of something that gets you to dependency hell faster. However, we shouldn't be advocating for committing the same sins of our fathers.
Anymore, as I evaluate fellow programmers, I'm looking for whether they've discovered "one more dependency" is like signing up for "one more subscription you have to remember to pay for" and what they do to try and mitigate it.
But as I got bit by the various issues with dependencies multiple times over the years, I have ended up preferring as few as possible and ideally zero beyond the standard library for hobby projects if I can get away with it.
If you only use a package manager for libraries that you have high trust in then you don't need to worry - but there are so few projects you can have high trust in that manual management isn't a big deal. Meanwhile there are many many potentially useful packages that can save you a lot of effort if you use them - but you need to manually audit each because if you don't nobody will and that will bite you.
One of the worst things working at companies shipping C++ was the myriad of meta-build systems that all tries to do dependency management as a part of the build system without having a separate concept of what a "package manager" is, this is truly the worst of both worlds, where people are happy to add dependencies, never update them, and never share code between projects and departments. I do not wish that way of working on my worst enemies.
Whatever problems package management brings is such a better problem to have than not having a package manager. That said I think everyone can get better at being more discriminatory of what they add to their project.
Yes, shared code has costs
- more general than you likely need, affecting complexity, compile times, etc
- comes with risks for today (code) and the future (governace)
But the benefits are big. My theory for one of the causes for Rust having so many good cli's is Cargo because it keeps the friction low for pulling in high quality building blocks so you can better focus on your actual problem.
Instead of resisting dependencies, I think it would be better to spend time finding ways to mitigate the costs, e.g.
- I'd love for crates.io to integrate diff.rs, provenance reporting (https://lawngno.me/blog/2024/06/10/divine-provenance.html), etc
- More direct support for security checking in cargo
- Integrating cargo-vet and/or cargo-crev into cargo
What a great quote.
> People abuse knives and stab people, so let's not have knives in our kitchen!
> People abuse package managers and create dependency hells, so let's not have package managers in our programming language!
No matter how you see it, this fits the definition of dumbing down;
Is this what you really want?
If that is the case, then we can shake hands and I will use a different programming language.
I am arguing that their "benefits" are only very short-term, if there is actually benefit for them in the first place. The strawman that you present has been repeated already and is not considering that all of those other things are actually useful and good alternatives.
I read you think these automations lead to more harm than good so are a net negative, and I understand that point of view
However, I think this is not a tautology
I think dependency hells and bad dependencies absolutely happen indeed, but are conditioned to badly managed programming projects
And I do not want to suffer from the dumbing down of stripping out package managers
And no, it's not a tautology, it's an empirical observation.
And as I said, not everything ought to be automated, especially hell.
Isn't this backwards? In real life, if you have a dependent, you are responsible for it. On the other hand, if you have a dependency on something, you rely on that thing, in other words it should be responsible for you. A package that is widely used in security-critical applications ought to be able to be held accountable if its failure causes harm due to downstream applications. But because that is in general impossible and most library authors would never take on the risk of making such guarantees, the risk of each dependency is taken on by the person who decides it is safe to use it, and I agree package managers sometimes make that too easy.
I mean, sure. So what does the solution look like? From my perspective it looks like a tool that is able to update your dependencies so that you can easily pick up bug fixes in your dependencies, which sounds an awful lot like a package manager.
> JavaScript is great example of this as there are multiple different package managers for the language (npm being one of the most popular), but because each package manager defines the concept of a package differently, it results in the need for a package manager manager.
This doesn't seem like a strong point to me. Yes, there are things like yarn, pnpm, etc. But IIUC practically all npm alternatives still define packages in the same way (a package.json at the root hosted by npmjs (or your private repo)), and the differences are ergonomic/performance related.
> [that each package manager defines the concept of a package differently] is why I am saying it is evil, as it will send you to hell quicker.
Then I think it's more of a language problem, not a problem with the concept of a package manager.
Exactly! Who has the time or the discipline to do that manually?
Nowadays it is mostly improved, and the others differentiate by enchantments to workspaces (better monorepo support) or more aggressive caching by playing games with where the installed packages physically exist on the system.
The core functionality- what a package is- has always been the same across the package managers though, because the runtime behavior is defined by node, not the package manager.
Obviously taking on fewer such liabilities?
> I am not advocating to write things from scratch.
and is clear in its target:
> That’s my general criticism: the unnecessary automation.
Yes, fewer dependencies is a solution, but it does not seem to be the author's position.
The "I am not advocating to write things from scratch" is more of a caveat to the people I know will comment NIH nonsense rather than anything productive.
But yes, my position is minimize dependencies and slow and carefully vet them too, and do not automate this process.
If only it where that easy.
Often the update isn't source compatible with the package that uses it so you can't update. There are some projects I use that I can't update because I use 6 different plugins, and each updates to the main project on a different schedule on their own terms - meaning the only version I can use is 10 years out of date and there appears no chance they will all update. (if this was critical I'd update it myself, but there are always more important things to work on so I never will in practice)
Sometimes a package will change license and you need to check the legalese before you update.
Sometimes a package is hijacked (see xv) and so you really should be doing an audit of every update you apply.
The solution IMO (which is non-existent afaik) would be to integrate some kind of third party auditing service into package managers. For example, for your npm project you could add something like this to your package.json:
` "requireAuditors": [ { "name": "microsoft-scanning-service", "url": "https://npmscanner.microsoft.com/scanner/", "api_key": "yourkeyhere, default to getting it from .env" } ] `
And when you npm install, the version / hash is posted to all of your required auditor's urls. npm should refuse to install any version that hasn't been audited. You can have multiple auditing services defined, maybe some of them paid/able to scan your own internal packages, etc.
I've thought about building a PoC of this myself a couple of times because it's very much on my mind, but haven't spent any time on it and am not really positioned to advocate for such a service.
You may not have the time to audit dozens/hundreds of dependencies pulled into your projects, but there's still something you can do. For Rust/Cargo, you can run tools that check every dependency against a vulnerability list. As you have source of dependencies, you can also run static code analyzers/auditors to scan for code smells, lack of unit tests, etc.
For Java, I use the OWASP plug-in of Maven to check dependencies for security vulnerabilities. I bet other languages' package managers/build tools have similar plug-ins.
Some auditing is better than none at all. You shouldn't do no checking just because you can't full auditing!
There are no solutions, only trade-offs. And the point is that not everything needs to be, nor ought to be, automated. And package managers are a good point of this.
And yes, a language with an ill-defined concept of a package in the language itself is a problem of the language, but the package managers are not making it any better.
If a language does not provide a definition of a package but a package manager _does_, then I would say that that package manager did make that aspect of the problem better.
> Through manual dependency management. Regardless of the language, it is a very good idea that you know what you are depending on in your project. Copying and vendoring each package manually, and fixing the specific versions down is the most practical approach to keeping a code-base stable, reliable, and maintainable. Automated systems such as generic package managers hide the complexity and complications in a project which are much better not hidden away.
So that makes all of us human package managers. It's also true that you can get a package manager from internet folk that works better than the processes and utilities your team cobbles together to ease the burden.
The Programmers’ Credo: we do these things not because they are easy, but because we thought they were going to be easy
Plus Lua for package recipes. It’s going really well!
Isn't that basically manual dependency hell?
Note to self... don't use Odin.
EDIT: I've been rate limited, so the point is: unless you're Terry Davis, you're not going to be able to write software of any real complexity. Few people are going to even bother to vet the standard library, let alone the compiler, the runtime, etc etc.
what if packages were meant to be read, and config was set inside the file directly
what if we transitioned to think of packages as templates, rather than generic black boxes
i think it would drastically reduce dependencies, package complexity, and improve understanding
NPM is even worse, you import one thing and get 1000s of trash libraries so nowadays the only JS I write is vanilla and I import ES Modules manually.
Also, Odin doesn't make adding dependencies that difficult, you can literally just throw an Odin library into your project as a folder and it's available. The Odin compiler does everything else for you.
But no, for the vast majority of people, SDL2 is perfectly fine, although SDL3 is a vast improvement. It's as stable and battle-tested as a cross platform multimedia library is bound to get. Opening a window and polling input is trivial.
Then again I've never even heard of the language they're using (Odin) so maybe that doesn't play well with a C library.
I'll just link to some of the bugs directly that posted as issues to SDL:
https://github.com/libsdl-org/SDL/issues/4789 (not fixed) https://github.com/libsdl-org/SDL/issues/4816 (closed) https://github.com/libsdl-org/SDL/issues/4790 (closed)
And these being the bugs we found ourselves, not other bugs that have already been found, and many marked "as not planned" since SDL2 is now finished.