Comment by lobster_johnson | Hacker News Reader

Comment by lobster_johnson | Hacker News Reader

Parent Full thread

lobster_johnson

The article is about vgo, a proposal (with working prototype) to build package management into Go itself.

vgo bundles several improvements, a major one being the introduction of "modules", which are versioned collections of packages. Modules, among other features, will let us decouple the import path from the file system structure and source repository, making the awkward and much-maligned $GOPATH finally obsolete.

vgo also introduces a new dependency resolution algorithm, minimal version selection (MVS), which is different from traditional dependency resolution algorithms in that it will at any time choose the oldest allowed version that satisfy the minimum version constraints specified in the list of imported packages, where other systems will choose the newest. As MVS does not require (as with dep) a complex boolean SAT solver with potentially pathological, NP-complete cases, it is much simpler and faster to compute. However, it also has downsides. (Edited, thanks to pa7ch for correction.)

The author likes pretty much everything about vgo except the MVS algorithm, which has been deeply controversial since the proposal was first published. The author goes into explanations of why he thinks MVS is a bad fit, but this is a complicated topic, so you really have read both the vgo proposal and this rebuttal to understand what it's all about.

The article was written by Sam Boyer, one of the designers/authors of dep. While he has been courteous about it, there are obviously emotions at play; during its development, dep seemed to have the blessing of the official Go team, and this proposal came out of the blue and probably felt like an ambush.

Minor correction: MVS requires modules to specify the minimum version to use of a module. The algorithm will then select the newest among those based on the idea that semver is forward compatible among minor versions. The name minimal version selection throws people off.

Other algorithms would select the newest release within a major version even if no module has ever tested it and it was released 5 seconds ago.

When authors do make incompatible changes and MVS selects a broken dependency, as expected, there are manual escape hatches. More complicated NP complete algo's would have more of an automated answer here by allowing dependencies to have boolean constraints (greater then X, less then Y, not Z etc.) on version so that there is this collaborative summation of constraints to work around authors violating semver or bugs. To summerize poorly: It seems Sam Boyer regrets that dep didn't do what vgo does by default, but believes that a SAT solver could kinda do MVS for the happy path but still solve against the introduction of boolean constraints.

Personally, I'd like to let vgo play out a bit, I'm not convinced boolean constraints are something I really want imposed on me by my dependencies.

More info for those who are looking for more context: https://research.swtch.com/vgo https://github.com/golang/go/issues/24301

Whether you agree or disagree, its novel research in this field and excellent technical writing worth reading in its entirety over a cup of coffee.

Thanks for the correction; I worded that unclearly. My point is that with MVS, the solver will favour the oldest package that is allowed by the semver version constraint; given transitive dependencies on multiple packages, it will conservatively pick the oldest that satisfies all the constraint. You can bump the version you want by updating the version constraint, but unlike other tools (e.g. Cargo [1]), it will always favour the oldest allowed version.

[1] https://research.swtch.com/cargo-newest.html

Well, sort of. It's more like it has implicit versioning based on a lock file.

In other systems like Cargo and NPM you have a "lock" file which specifies the exact current version of dependencies to use, e.g. if you say you depend on "foo >=1.0" and when you build your project it actually downloads "foo 1.4", this will be written to the lock file. When foo 1.5 is released, it won't be automatically upgraded even though you said it should be fine.

vgo is exactly the same. When you first add foo1 it will implicitly (via semver) assume you meant "foo >=1.0" and download the latest version - foo 1.5. That version will be written to the lock file (go.mod) and when foo 1.6 is released it won't be automatically used. Exactly the same as NPM and Cargo.

And just like NPM and Cargo you can trivially update to the latest compatible versions of your dependencies with `vgo get -u` (like `cargo update`).

It's not that different. The main differences are you can't specify maximum versions, and different major versions are treated as different packages so you can easily have them both.

MVS has another quirk, which is that there are no version ranges. Only minimum versions.[1]

It relies completely on the community adopting "Semantic Import Versioning", which is basically "v2 is any breaking API change, which must live along-side v1 because it's actually a totally different import". So it blends some of the pros and cons of both dep/cargo-like package management with those of npm's.

It's... interesting. And I love that it's not an easy answer either way - it's a significant alternative, and it's bringing up lots of new(ish) ideas and approaches that have kinda fallen on the side. That said, I've had concerns about vgo from the beginning, primarily around the "does this make sense in an ecosystem". Package managers are not just "this algorithm is efficient", they have major code-culture effects (and the ecosystem can support or destroy the package manager's goals). This article is hitting most of my concerns squarely on the head, in quite a bit more detail than I've managed to figure out.

Also, https://medium.com/@sdboyer/so-you-want-to-write-a-package-m... is very good. sdboyer has been thinking about this stuff for quite a while - emotions may be there, but he has a LOT of experience backing it up.

[1]: To work off / echo pa7ch's comment here: it picks the highest minimum when there are conflicts in transitive dependencies. So if you use A (which needs C v1.1+) and B (which needs C v1.3+) you get C v1.3, even though v1.4 is available. You can upgrade to v1.4, but only by putting "C at v1.4" in your top-level dependencies list (which the tool automates for you) to override those values, and it's assumed safe because it's still importable, therefore still v1-compliant.

Reply on news.ycombinator.com