Could you contrast Cargo and vgo for those who only know the former?
* Semantic Import Versioning. A package is identified by some name. I think vgo calls it an "import path", but I have also seen "package path" used. Every version of that package must remain compatible with previous versions. You may not break compatibility (i.e. increment the major semver number) without also renaming the package. There is some syntactic sugar that makes it clear that a version 2 is closely related to the original version 1, but from the point of view of vgo, the two versions are completely distinct packages. This also means a program may depend on several major versions of the same package (which Russ claims is necessary when doing gradual migrations of large programs).
* Minimal Version Selection. In contrast to pretty much every other package manager, vgo tries to install the oldest version of a packages that still satisfies any lower bounds specified anywhere in the dependency tree. Upper bounds are not supported. This both makes the solving algorithm trivial (which probably doesn't matter much), but also gives you reproducible builds without using a lockfile (which is nice), and it gives you a very simple operational model for what the package manager is doing (which I think is crucial).
All package managers suck and all package managers break, but I think when vgo breaks, it will be more obvious what is wrong. Time will tell whether it works in practice!
(And I'm not even a Go programmer; I just like the thoughtfulness that goes into the tooling.)
That's really interesting. Obviously this would lead to fewer regressions, but it seems opposed to the "security problems fixed for free" scenario you hear about with other package dependency schemes. Were we all just imagining that scenario? Is there another way we should be handling vulnerabilities, i.e. by "repudiating" old versions rather than simply releasing new ones? Then you could have something like "Minimal Unrepudiated Version Selection".
You can't have it both ways.
You either get reproducible builds (by default with vgo, by adding lock files in other systems) or you get silent upgrades that potentially fix security issues (but also potentially introduce security issues).
You also mis-characterize the position of vgo creator. He doesn't think that lock files are a huge issue per se.
The difference is in default behavior of the system: vgo by default picks predictable, consistent version of dependencies. That version doesn't change if dependencies release new versions.
In fact, it's not just the default behavior but the only behavior.
Other systems allow specifying complex rules for which version of dependency to pick and they all allow for a scenario where you run the same algorithm over the same rules but pick different version because in the meantime some dependency has released a new version.
It's such a big problem that all those system end up introducing lock files, which is a tacit admission that what vgo does by default is a good idea.
vgo doesn't need lock files to get the benefit of lock files, which is a nice cherry on top but the real advancement is in changing the default behavior of how resolving versions of dependencies work.
That is the great thing about lock files. You don't have to check them in. If you want reproducible builds, you do check them in. If you don't, you don't. The user, not the package manager, gets to make this choice on a case-by-case basis.
> The difference is in default behavior of the system: vgo by default picks predictable, consistent version of dependencies. That version doesn't change if dependencies release new versions.
And that right there is the problem. You don't get the latest version unless you explicitly ask for it. This makes the user do what a package manager is perfectly capable of doing. It thrusts the problem onto the user instead of applying a well-known solution that was causing problems for nobody.
If you are writing an application, don't you want your transitive dependencies to get security fixes without having to trawl through the tree?
And how does it guarantee that this actually results in anything that works, if upper bounds are not supported?
The "known good set" of packages as used by buildout for any given plone version were not really easy to work with.
See eg:
https://docs.plone.org/4/en/manage/troubleshooting/buildout....
Which points to things like: http://dist.plone.org/release/3.3.5/versions.cfg
Which aren't that much worse/better than a gemfile.lock for a rails project, I guess.
Now, I guess the better answer is to have smaller projects, with different interfaces (eg: don't give all modulles/parts access to your zodb object database, have a ome json/rest interfaces etc).
But apparently there are still many "big systems" being built.
This alone would make npm 100x less painful to use.
Admittedly, the "update all packages to latest compatible version" flag described above sounds very nice. There is approximately zero chance the npm developers would ever accept such a PR, but there would be nothing wrong with a utility that accomplished the same thing.
> vgo tries to install the oldest version of a packages that still satisfies any lower bounds specified anywhere in the dependency tree
Another benefit of this that may be a little less obvious (it wasn't obvious to me) is that this means every package-version selected by MVS in the entire dependency tree is explicitly specified somewhere in the dependency tree. This means that every selected package-version was specifically tested with at least one other package (assuming, reasonably, that libraries are tested with the versions that they specify). You don't get this with SAT, in fact it's possible for SAT to select a set of packages where no two package-versions were ever tested with each other anywhere else before.
I feel that this property, combined with upgrades requiring affirmative action by the root package maintainer (by editing go.mod or having a tool do it), will make vgo-managed packages much more reliable over time.
I find the claims that vgo removes the lockfile disingenuous. Yes it's technically correct in that there's no lockfile, but it's incorrect in that you've basically turned the file that declares dependencies into the equivalent of a lockfile.
With something like cargo, I can `cargo update` and it will fetch new versions of my dependencies and update my lockfile. Net result, I have one file to commit changes to (the lockfile), with zero manual edits.
With vgo, updating a dependency requires manually editing the dependency list to specify the new version. Net result, I have one file to commit changes to (the dependency list), but it required manual editing.
In both scenarios, the one file that I have to commit changes to specifies the version of the package that will be used. Though that's not actually strictly true with vgo; there it specifies the minimum version that will be used, but that's not necessarily the actual version if another dependency requires a later version. Not a big deal, but it does mean there's no single file I can inspect to find out what the resolved package version is.
So that difference isn't actually all that different.
This is somewhat misleading. Most package managers use a human-edited manifest containing constraints and a machine-edited lockfile, containing the chosen versions. You periodically (or after changing dependencies to require newer versions) run a tool that fetches the newest versions satisfying your constraints and writes out a lock-file, which you then commit and use at install-time to get reproducibility.
Go modules use a single file that serves as both and is both human- and machine-edited. You periodically (or after changing dependencies to require newer versions) run a tool that fetches the newest versions satisfying your constraints and updates the manifest with them. The manifest is then (also) used at install-time with minimum version selection to get reproducibility.
What that means is that in practice, Go modules never install older versions than the equivalent in other package managers would do. The maintainer specifies constraints and uses a tool to decide what version will actually used to at build-time. In both cases, that tool will choose the newest versions available.
What is correct (and contentious) is that Go modules have no concept of upper bounds for dependencies. So the complaint is, that as a library author you can not prevent a newer version from being chosen by one of your reverse dependencies. It remains to be seen how much of a problem that will be in practice.
So, if anything, the problem is that as opposed to other package managers, Go modules sometimes choose too new versions, from the perspective of some people. It never chooses too old versions.