The pain that minimal version selection solves
about.sourcegraph.com
about.sourcegraph.com
But it doesn't solve the "A dependency of a dependency has a massive security hole in it, and there's a patched version which has just been released" pain.
Which I'd argue is more important.
For example, introducing automated systems which update dependencies to more recent versions, run automated tests (you already have CI set up, right?), and notify authors (including a PR; why not?) when versions can be bumped automatically.
This sort of system A) doesn't involve a SAT solver and B) gets things up to date all the time.
It's also roughly analogous to the principles that `go get` has already forced into the norm in the community: things should build and work against master in the dependencies; if they don't, you've got problems. At the same time, since precise dependency versions are tracked, changes can be absorbed at a controlled rate. What a trade!
I've argued for this in various forms for years. Regardless of how the dependency version is calculated, having that version chosen outside the context of version control is asking for trouble.
The reason that maximal version selection existed in the first place was that manually updating dependencies has been seen as too hard, so it gets punted to the tool. The net result is that the code in production can't necessarily be associated with a particular dependency.
Of course there's a cottage industry for doing this automatically, as their should be. In the case where proper CI and automation exists, I feel like minimal selection has a slight edge in that it is less likely to click unexpectedly upwards.
But in practice, the practice of automation makes the overall policy of version selection close to irrelevant.
Automate automate automate. Never leave this stuff to chance.
This may not be a problem in Go, where projects don't have that many dependencies and dependency trees are usually very flat, i.e. dependencies don't often have subdependencies¹. But in the JS ecosystem an average application has thousands of dependencies, many levels deep (as it's super common for libraries to have many subdependencies), so the effect on the ecosystem would be huge. And if there are more old versions around, that creates more maintenance effort for maintainers (you cannot avoid this with policies no matter how hard you try, proved empirically by having been an open source maintainer of many projects).
¹As to why this is, I think it is partly because Go dependency management has been so bad - you can't really depend on other packages in your library if there is no standard way to declare them. Better and more standardized dependency management will likely also make people rely on dependencies more (especially in libraries, so the tree will become deeper). But it's also some culture in Go I've observed that I've heard being described as "copy+paste oriented programming". Go is more verbose than other languages and people don't have a problem with duplicating code that much. And the Go standard library is enough for most use cases - on the other side, native JS APIs are notoriously bad (especially historically with browser differences etc), so relying on libraries for everything is completely natural (you're weird if you don't).
Not necessarily. Who said that we're honoring constraints every single library declared for its transitive dependencies?
We could. But perhaps we should be careful not to assume all systems have to be built this way.
> in Go ... dependency trees are usually very flat ; in the JS ecosystem ... many levels deep
Right. So if we made the choice by fiat to simply disregard version constraints proposed by libraries for their transitive dependencies, and determine our own version selections purely by testing satisfaction of interfaces and tests, this would work out very differently in these two ecosystems due to those coefficients. In one of them, it's quite tenable in practice.
MVS has this shortcut without requiring a lockfile. Just update the minimum version for that dependency in your own project.
The "security issue" argument is bogus because while new code can fix bugs it can also introduce new bugs.
Every competent dev shop has code reviews.
A code review exists because you don't trust your coworkers to not introduce bugs.
But somehow you're ok with people you don't know to effectively check in code into your code base without as much as running a smoke test.
Re-reading the thread, maybe this is what you meant? I'm not sure, possibly because this is also what 'kjksf meant.
The answer for both is that version updating should trigger CI activity and then be automatically applied when tested. Dependency tools should be as strictly dumb as possible.
This gap exists because dependency management tools blossomed and matured in a manual developer-at-a-keyboard context before the kind of automation that makes their clever bits less relevant or necessary.
I don't follow. Could you elaborate?
Old versions only ever increase their known-bug count.
Only a new version can reduce a known-bug count.
It follows that the longer you stick to a version, the more problems you are guaranteed to be wilfully ignoring.
There are other reasons that automatic version selection at execution time (whether minimal or maximal) are problematic. But the idea that security is "bogus" is a bit of a stretch.
I thought that was obvious from context. I guess not.
I'm used to the idea that latest versions are safer, mostly because I have worked in and around systems intended to create that assurance. Where I work we track every upstream CVE patch that affects us until it has been released to customers. Large parts of this process are automated. On a good day the patch lands upstream and a fresh version of our own software rolls off the production line a few hours, fully baked and fully tested.
My sensitivity to allowing versions to fall behind comes from knowing that a) it doesn't have to happen and b) that software that fell behind cannot reduce its bug or vulnerability count without being upgraded. New versions absolutely introduce new bugs, but on the balance of risks, a known vulnerability is more likely to exploited than an unknown one.
I was/am confused because the context is max selection vs MVS. You appear to be arguing against MVS, so I assumed you were taking the max selection position, but max selection is incompatible with patched versions while MVS is not.
Based on your most recent comment, it appears you're not arguing for max selection nor against MVS, but rather you're arguing against pinning to an unpatched version (I don't think anyone in this thread is advocating for this, hence my confusion).
It's still unclear to me if you're arguing for max selection or MVS with latest patch version. Your third paragraph seems to set up a false dichotomy between max selection and unpatched old versions while your second paragraph seems to favor MVS/latest-patch-version.
This whole conversation seems to be unnecessarily complex with its digressions and false dichotomies. I think everyone can agree on the following:
1. Pinning to unpatched old versions is the worst of all (no one is advocating for this afaict)
2. Max selection is susceptible to new vulns (the rate of vuln discovery is greater in new versions than in old versions)
3. MVS/latest-patch-version (go get -u patch) is the safest of the three
The core issue that caused all of OPs problems was that Renovate Bot claimed to have only updated one dependency (@sourcegraph/codeintellify), but also updated another, unrelated one (sanitize-html). If you can't trust the tools around you not to lie to your face, all bets are off.
It doesn't matter whether our tools follow MSV or the opposite (LSV – Latest Version Selection?). We should always pin our dependencies to exact versions, and explicitly update them as needed. Renovate Bot (and its alternatives) attempt to automate that part. They send you notifications or even open pull requests, you can inspect the changelog of your dependencies and decide for yourself if the update is necessary (eg. because of security reasons) or if you prefer not to update out of fear of breaking something. Right now it seems Renovate Bot sucks at that, and should be improved accordingly.
Maybe it /should/ do something special though. Maybe it should stop using npm.
If it says "I updated package X" and it really updated X /and/ Y, then it didn't do what it promised. Maybe it should say "I updated package X and maybe also others I don't know…" that would be more accurate to how it currently works.
There's no way to solve this problem mechanically, other than to blindly trust the upstream that it only fixes security issues. They could just as well introduce ones (even in a patch that fixes an older one).
Instead, we could have tools that check our "freeze" files, and get advice (from a community resource that combines upstream notices "e.g. my package xxx < v4.5 is unsafe" and community voting/tagging) to give advisory ("you better update this package").
As kjksf already stated, patch versions can introduce security vulnerabilities too.
This is just a terrible argument. Sure the problem is NP-Complete to find an optimal solution for the general case, but you don't need to do that. You need to find a good-enough solution for the cases that people actually use. Heuristics for that problem are plenty fast and even finding the optimal solution is fast enough for most of the problems people are actually trying to use.
(Not that I'm saying individual homes are a good idea in general, but the NP-complete argument is pretty dumb).
Is Version-SAT FPT? (Fixed Parameter Tractable)
>Renovate Bot noticed that there was a new version of codeintellify so it helpfully created a pull request to update our main repository to codeintellify 3.6.0.
while this sounds really cool and I'll check it out later, automatically updating your code just because a dependency has a new version seems like a REALLY bad idea.
The value here is that it notifies you if an update is available (by opening a pull request), automatically tests the new dependency (when CI runs on your pull request), and makes it easy for you to apply the update if you want (by merging the pull request).
Unexpected in the sense of “oh my puppy chewed up the table leg”? No, this is pretty much expected behavior from npm.
Unfortunately Yarn has a bug in it that I don’t know how to fix. There’s a lot of stuff going on in both code bases and I can’t help but feel like they’ve exceeded their intrinsic complexity via coupling, and that’s why people are struggling to get them to behave correctly.
I wonder if splitting this job up into separate tasks and working on them separately would net some sanity.
Do any of Yarn, Bundler, or Cargo have this behaviour?
>People perfectly understand and apply semantic versioning (see also: compatible versioning).
Doesn't any dependency management tool assume that some form of versioning is followed strictly? Go modules have another assumption besides that one, the Semantic Import Versioning principle.
I understand OP's pain, but it seems to stem from bad design decisions made by NPM authors, and doesn't have a lot to do with Minimal Version Selection.
> it seems to stem from bad design decisions made by NPM authors
Yep.
> doesn't have a lot to do with Minimal Version Selection
The pain I experienced has nothing to do with MVS (because npm doesn't use MVS). The observation of the post is that my pain would not have happened with MVS. No more, no less.
I don't see where that assumption is baked into npm - the creator of a package declares the dependency constraints, so they are the ones making the choice which versions can be trusted well to work. You could pin all dependencies but then users would need to download tons of duplicate dependencies because of slight version differences everywhere. So it's a tradeoff, but the user makes it, not npm.
npm’s design incentivizes projects to assume “patch versions are safer to update than minor versions are safer to update than major versions” to avoid dependency duplication.
Such incentives do not exist with MVS because all that is required is a total ordering and there is no duplication within a major version of a module.
It would probably be very difficult to set up, but I imagine it could start by testing the minimal version, and if that works well, then do a binary search to find the latest versions of all dependencies that still work well. In the case of an exception stack trace, it could perhaps find the offensive dependency quickly, but even if not, a brute force binary search should still work.
We've all been in the situation where we fell behind some dependency and, upon trying to upgrade, found that we were so far behind that everything was broken in this-will-take-weeks-to-fix ways. That kind of work tends to get shunted into the too hard pile, where it generally only gets harder. Eventually the codebase ossifies around a very particular dependency set that can't be upgraded without stopping feature development.
The fact that it may be carrying a stew of published vulnerabilities tends not to be enough to trigger a change: work to upgrade is concrete, immediate and costly. Exploited vulnerabilities are probabilistic, sometime in the future and not taken from the current budget.
In practice stuff is kept up to date by rigid process discipline or by automation. Of the two, automation is more likely to be successful and sustained.