Go vgo: Semantic Versioning and Human Error | Hacker News Reader

Go vgo: Semantic Versioning and Human Error

codeengineered.com

131 pointsaleksi68 comments

Preface: vgo _specifically_ calls out the fact that maintainers of libraries have to be backwards-compatible within a major version, and that the onus is on users to put their trust into libraries not to break them appropriately:

> Modules are assumed to follow the import compatibility rule—packages in any newer version should work as well as older ones—so a dependency requirement gives only a minimum version, never a maximum version or a list of incompatible later versions.

(https://research.swtch.com/vgo-mvs)

Back to the article - it seems predicated on this scenario:

> “Our project depends on X@v1.5.0 right now, but it doesn’t work with X@v1.7.0 or newer. We want to be good citizens and adapt, but we just don’t have the bandwidth right now.”

If your deps + your transitive deps for some package are:

- 1.5.0 (you)

- 1.5.1 (some transitive dep)

- 1.4.7 (some transitive dep)

vgo will choose 1.5.1.

However, if your deps for some package are are:

- 1.5.0 (you)

- 1.5.1 (some transitive dep)

- 1.7.3 (some transitive dep)

vgo will choose 1.7.3 and presumedly your app will break.

In other dep managers, you might specify <1.7.0. How would this work? Grab two versions of the package (1.5.1 and 1.7.3), rewrite the import paths of the stuff that requires 1.7.3, and kind of opaquely have two version of the same thing? Or perhaps modify the way the import "xyz" works to be more opaque to solve this problem somehow? There's no nice solution to this.

This seems a fairly reasonable tradeoff; on the upside is a _very_ fast, very simple, and very predictable dependency manager. On the downside is that I have to really think about which libraries I trust not to break me instead of relying on my tool to specify ranges and the like.

Generally though, the ask from vgo is that folks care about backwards compatibility and think about trust, rather than covering up the issue. It's not going to be great for everyone, but I like the straight forwardness of it.

Other dep managers could at least tell you about the conflict. Maybe that's what vgo needs, a way to specify a conflict bound so that any downstream user is notified if the depgraph has conflicts (ofc conflict bounds would only apply after del resolution as simple checks otherwise it wouldn't be MVS)?

> Maybe that's what vgo needs, a way to specify a conflict bound

This is a valid idea; you should make a proposal or issue for it. Alternatively, it seems like something that would be easy to add separately - I personally like the idea of vgo as more of a simple, unix-like building block that you'd build on top of. Adding a small utility CLI checks incompatibilities with `vgo list` + some incompatibilities.txt file would be pretty easy.

> Grab two versions of the package (1.5.1 and 1.7.3), rewrite the import paths of the stuff that requires 1.7.3, and kind of opaquely have two version of the same thing?

Yes, in systems like Cargo you end up with multiple versions of the same package. This typically "just works". It works so well, in fact, that often times people don't even realize they're using multiple versions of the same package, and they want Cargo to report an warning here.

> This seems a fairly reasonable tradeoff; on the upside is a _very_ fast, very simple, and very predictable dependency manager.

For other package managers, speed of the core dependency resolution algorithm has never been a problem for me or anyone else I know.

Go packages can have side effects. For example a global map, background goroutines, shared connection pools, etc. Including multiple versions of the same package would not work for these cases.

Actually this isn't really a go specific thing. If I have a struct defined in a package version 1 that gets an extra field in version 2 how can I possibly reuse that struct between packages.

Because every other dependency management system has realized that nothing about the version number actually tells you anything about compatibility & so they don’t even try.

That vgo encodes semver as sone sort of contract system is crazy pants on a whole new level.

Sure they do. What if I do want to share the same struct between packages? I need both of my dependencies to share the same version of their dependant packages.

I don't think I understand what you're getting at. In ruby I can say I depend on a version > 1.2.0 and some other project could use my package and use version 1.2.1, thus altering my dependency. Bundler lets me do that because it makes assumptions about what version numbers mean.

The lock file makes sure this only changes when I want it too, but semver is an integral part of the system

Many package managers interpret semver in a similarly mechanical way, e.g. npm, bundler, cargo (and, I think, elm-package, and I'm sure others too), but they choose different versions based on that information.

And crucially, in these systems you are given an escape hatch (declaring versions that are not ok), which vgo does not support.

I think vgo allows you exclude specific versions in go.mod. It only works for specific versions, not ranges though

There are exclusions, but they are only available for the current module:

> Exclusions only apply to builds of the current module. If the current module were required by a larger build, the exclusions would not apply.

https://research.swtch.com/vgo-tour

> Go packages can have side effects. For example a global map, background goroutines, shared connection pools, etc. Including multiple versions of the same package would not work for these cases.

Yes, that's a problem too, but there's a lot of packages which don't have side-effects (I'd expect more of the latter, in fact).

Your point that there needs to be control over packages that should be singletons and/or "public dependencies" is a fair one (it's something that has been considered for cargo for a while, but I'm not sure it's been implemented yet), but allowing multiple versions for packages without side-effects is a strict improvement over the abstraction-breaking "one version globally" for every package.

> If I have a struct defined in a package version 1 that gets an extra field in version 2 how can I possibly reuse that struct between packages.

You can't: they're different types. The underlying assumption of having multiple versions of a package is that imports have unique (name, version), not just name, meaning "a" v1 and "a" v2 are treated as completely different packages, just like "a" v1 and "b" v2.

So what's happened here is there's a bug in in gRPC 1.8.

What the OP wants is a way to say, "hey, our software is broken with this buggy version of gRPC," as part of the dependency requirements. I can see some value in that, but it also feels like a case where just documenting it in the README would not be super-unreasonable.

As a practical matter, any time a dependency A says it wants some other dependency B at version V, if you are using B at a version U>V, you are running in a untested configuration, and may encounter problems. (But! this will only happen if you are specifically requesting U>V, or if you have some other requirement A2 requesting B@U. In the latter case, again, you are running a new, untested combination of software, and there might be problems.)

An alternative to this would just be allowing multiple copies of B at the same major revision, B.U and B.V. But due to Go's use of package-level global variables, that is just as likely, if not more so, to cause bugs, since not all packages will see the same set of shared variables.

It does seem like extending the vgo to allow != version requirements might be reasonable. Not full-on <=, which breaks builds or leaves them unsat forever, but just a != to say "hey this version is buggy and we know we don't work with it."

> So what's happened here is there's a bug in in gRPC 1.8.

Yeah I don't understand how this is vgo's problem.

In the follup post it shows that they are importing the buggy gRPC version directly: https://codeengineered.com/blog/2018/golang-vgo-broken-dep-t...

So doesn't that mean that they can just file a bug report upstream to get the regression fixed in 1.8.1, and exclude 1.8.0 in the meantime?

I agree. I don't see a vgo issue.

The behavior in the example was broken by a release, but could have as well been restored by a patch release as restoring behaviour would not have broken the API.

See https://news.ycombinator.com/item?id=17123583

For vgo there is an explicit goal is to avoid NP-completeness for some reason. Systems of not-equals are NP-hard (trivially models 3-colouring), so allowing not-equals constraints on package versions is a non-starter if the stated goals of vgo are to be maintained.

Personally I think that some kind !=-requirements are needed for package managers just for this case, and thus the choice of dependency resolution approach vgo has is wrong IMHO.

One possible 'solution' to this is to disallow publishing minor versions that break the api, basically compare all the public signatures to see if they change. E.g. adding a new field is ok, changing a field type is not, adding varargs parameter is ok, changing the order of parameters is not etc etc.

The problem with this solution is that behavior is part of the api, but it is not encoded anywhere that can be compared automatically.

That's exactly what happened in the example in the linked article. Two new fields were added (no problem, that's allowed), and the implementation of another field was changed. This change is either fine because it doesn't change the behavior (refactoring, optimization, fix a bug, etc) or not fine and it changes behavior that breaks existing code, which is what happened in the example.

This behavior was not encoded anywhere, so an automatic publishing gate wouldn't have caught this.

It’s not just public signatures, its also things like types—even private types. For example, if public struct type Foo has a private member of private struct type bar, and private type bar is changed to add a field, this changes the public interface because the memory layout of the type has changed.

eh... does this matter though? Go uses static linkage, you have to recompile your code when you update third party libraries anyway.

It can cause behavior changes. Especially around memory usage & gc.

That is not longer the truth for quite some time now.

Yes, Go uses static linking by default, but it also supports dynamic linking and also has a plugin concept built on top of it.

You are right: signatures do not encompass behavior. Some other examples I have observed:

What if the implementation of some (signature-identical) function changed its performance characteristics dramatically, such that it was no longer valid for use in certain scenarios (i.e. something that previously checked the filesystem now makes a costly query against a remote database)?

What if a function that was previously thread-safe changed, and became non-thread-safe?

What about changes in space (memory) consumed by a function?

These aren't rare classes of behavior change, and they certainly aren't rare pathologies for bugs. And, at least in Go, there's no easy way to encode them in the signatures and/or memory layout of your package in such a way that two non-identical versions can be tested for compatibility automatically. These aren't edge cases.

The behavior in the example was broken by a release, but could have as well been restored by a patch release.

See https://news.ycombinator.com/item?id=17123583

I don't see a vgo issue.

One way to enforce this is to never do manual releases. There's a package / standard out there called Conventional Changelog, which enforces a certain style for commits (notably, putting the type of code change in the commit), and which allows a script to automatically change the (semantic) version of a package - if there is any commit with a backwards incompatible change, it'll make it a major release.

Of course, this still depends on humans to type the correct commit message. Especially with Node libs, and I can imagine the same is true for go libs, there's no four eyes requirement, little to no review before a version is released.

vgo was explicitly designed to balance out two needs: (1) the need to use known good versions of the dependencies, and (2) the need not to burden dependency consumers with meaningless constraints (especially with an upper limit on the version). This article shows* why the first need is important, but it does not give vgo the credit for satisfying it with its minimal version selection (and, indeed, for providing a more stable and hence reliable solution than maximum version selection), and it misses the value of the second need. In my experience, the upper limit on the minor version is most often arbitrary, and in some cases when it is not, a future minor version reverts the mistakenly introduced incompatibility. Therefore vgo approach has unique advantages over other version selection methods, and it should not be discounted for the lack of a feature necessary to provide them.

* The article says that "Prior to 1.4.0 there was one function of MaxMsgSize" which "had previously set the size on both send and receive" but it does not substantiate this claim, and it may be false since go-grpc 1.3.0 documents that "MaxMsgSize returns a ServerOption to set the max message size in bytes for inbound mesages" https://github.com/grpc/grpc-go/blob/v1.3.0/server.go#L166, and it has not changed in go-grpc 1.12.0 https://github.com/grpc/grpc-go/blob/v1.12.0/server.go#L228 which strongly suggests that this is not a bug.

> it does not give vgo the credit for satisfying it with its minimal version selection (and, indeed, for providing a more stable and hence reliable solution than maximum version selection)

I did not argue that because it doesn't do that. In an effort to not spoil things, because Sam Boyer has done a lot of work on this and wants to write about it, I won't say to much.

MVS does use a maximum. It's the major version number in SemVer. It's implicit. You can't override it when dealing with transitive dependencies. For it to always be a safe value people have to always follow SemVer. Unfortunately, they don't. In a perfect world this would work. Unfortunately, people are fallible and we need a system that works in light of that.

vgo trades some safety (by not supporting upper bounds) for some utility (by not artificially limiting the lifetime of a released library). Yet package managers that do support upper bounds do not guarantee safety, because libraries may not specify upper bounds or they may specify too broad bounds. This is a trade-off, and I have not seen anything convincing about why vgo position on this trade-off is unreasonable.

Can you give an example where vgo prevents use of a library where another approach does not? The main difference in the expressive power between vgo and traditional approaches is that the latter can restrict your use of libraries together more. vgo does not need a perfect world: it is practical in the imperfect one.

> MVS does use a maximum. It's the major version number in SemVer.

So, we can not force a library that wants dependency v3 to use dependency v2 (and vice versa), even if the author of the library knows that it works with either v2 or v3. This is a loss of vgo. On the other hand, if another library can only work with v2, and yet another can only work with v3, vgo allows the use of both in the same application. This looks like an acceptable win for the price of that loss.

Just had this issue recently with a Python library. It claimed to be "out of beta". A month later they broke half the public API moving from 0.3.0 to 0.4.0. Although to be fair, this project didn't claim to be using SemVer.

Breaking API between 0.* releases conforms to SemVer: https://semver.org/#spec-item-4

Interesting, I must have overlooked that before. Thanks for the info.

You are claiming that is a one year old issue in the grpc-go project.

But we are not seeing any change in that code (https://github.com/grpc/grpc-go/blob/master/server.go#L228) in the following releases. And I tried to look for an issue mentionning MaxMsgSize and found nothing. https://github.com/grpc/grpc-go/issues?utf8=%E2%9C%93&q=MaxM...

The fix that would restore the compatibility you claim seems obvious:

  	func MaxMsgSize(m int) ServerOption {
  		return func(o *options) {
  			o.maxSendMessageSize = m
  			o.maxReceiveMessageSize = m
  		}
  	}

This fix could have been applied in a minor release to restore compatibility without breaking the API.

So if this is an issue, why is this not reported?

> In my experience, the upper limit on the minor version is most often arbitrary

In my experience, this is the dependency version that was used when testing the library depending on it. As soon as your tool swaps it out for a newer version, you actually run an untested combination. Yes, it should work. But as we all know, it often does not.

And then the tool does not even have a proper feature to enable you fixing it on your side (e.g., by pinning a whole dependency tree).

> And then the tool does not even have a proper feature to enable you fixing it on your side (e.g., by pinning a whole dependency tree).

vgo allows you to pin your transitive dependencies to the exact versions of your choice, as long as non of them require a dependency with a higher version than you prefer. (But then, do other dependency managers let you disregard version constraints of your dependencies?)

AFAIK vgo only allows me to pin individual packages, but not whole trees. How would I pin a dependency tree in vgo?

You can copy the output of "vgo list -m" (the list of transitive dependencies with the selected versions) into the "require" section of "go.mod" and increase the versions that you want to change. (The next invocation of "vgo verify" will delete the lines with versions that you did not change because they are implied by the lines with versions that were not deleted.)

thanks for the detailed explanation. That would at least provide a workaround.

For the uninitiated, vgo has a way to blacklist/exclude certain versions of dependency modules, but it only works for the top-level module being built. Exclusions from deeper dependencies are not honored in order to avoid an np-complete satisfiability problem (https://research.swtch.com/vgo-mvs).

Two quick thoughts...

1. By not automatically hoisting that information there is a requirement for the developer to know those details for all transitive dependencies and handle those issues themselves. A task that was automated under dep (and in other languages) is now a manual task in vgo. I do not look forward to doing that when I import Kubernetes into a project as a dependency.

2. Writing a solver that can work over a rather large codebase (e.g., kubernetes) in a timely manner (as fast as those for other languages) can and has been done. Why are we trying to avoid satisfaction problems, whom do they benefit, and why?

Dependencies' exclusions are not honored, but what if an exclusion produces a warning or notice if MVS happens to select that exclusion?

That would help solve the problem of you not knowing that a later version inadvertently breaks a dependency without taking away your (as the top-level module maintainer) choice.

> The issue Sam called out about handling the case where someone isn't following the spec. This can be on purpose or by accident. It's not uncommon to find it.

This is a common occurance. So common - in fact - that I think it’s almost always better to either pin everything or nothing and hope the unit test will find any breaking behavior.

This is a slightly tangential issue to pinning. With MVS and vgo you cannot set a maximum version. When the resolver walks the tree it doesn't know when a version is too new and could break things. Even if it pins it could pin an incompatible version.

Maximum version information is not communicated.

Note, I'm the posts author.

Matt:

> This is a slightly tangential issue to pinning. With MVS and vgo you cannot set a maximum version. When the resolver walks the tree it doesn't know when a version is too new and could break things. Even if it pins it could pin an incompatible version.

This just isn't true. The number you put in the go.mod file can't encode the max version: true. But it won't change underneath you so when you run "vgo list -m" and determine the solved version, it won't change from that.

go.mod isn't a lock file. It doesn't contain the entire dependency tree the way other lock files do unless you choose to track, at the top level, the entire dependency tree yourself. There aren't tools to prune or update that for changes in the underlying transitive dependencies, either.

It's not a lock and from what I gather from Russ not intended to be.

The solved version doesn't mean it's the right or even a compatible version. This issue here is no maximum version meaning it could solve for an incompatible version due to not having all the information about the complete tree.

go.mod paired with MVS effectively locks versions into place. It must be computed, but it effectively locks all versions into place for reproducible builds.

> It's not a lock and from what I gather from Russ not intended to be.

go.mod files lock versions into place. If you disagree read the code / design documents. I find it dishonest to say something technically true "It's not a lock [file]" and yet not be true, as it together locks versions in place.

> it effectively locks all versions into place for reproducible builds

Assuming git tags don't move.

Modverify is optional and detects that.

You're right, MVS is deterministic without a lockfile, which is what you're asserting it solves.

The issue it does have is one where a transitive dependency is known bad by one of your immediate dependencies, but there's no way to declare that to the resolver.

Here's a concrete case:

1. I depend on cool-framework, min 1.5.0

2. I depend on boring-library, min 1.0.0

3. cool-framework depends on boring-library, min 1.0.0

4. cool-framework receives bug reports that boring-library 1.5.0 breaks cool-framework.

5. I upgrade boring-library to min 1.5.0. Everything builds and tests okay, but I get breakage in my staging env.

There wasn't non-deterministic package install behavior, but I still ended up with breakage that could have been prevented after (4) in a system that allows library dependencies to articulate more complex version constraints than "min version". In Rubygems or Python, with or without a lockfile, cool-framework could update its dependency on boring-library to specify "min 1.0.0, less-than 1.5.0" until the breakage is fixed.

Ok so what happens when boring-library v1.5.1 is released that fixes the breakage, right after the maintainer of cool-framework goes on vacation? Now you have the opposite problem: you can't update to a perfectly compatible version because one of your dependencies can enforce arbitrarily complex constraints on which libraries you can include. Now the problem isn't a simple "oops gotta revert the upgrade until later" which either library can fix, now the issue is that one of your dependencies is blocking all upgrades for (now) no reason.

There are problems either way. I think the other benefits that MVS enables is worth choosing one of these problems over the other.

Edit: As mentioned upthread, vgo supports exclusions for the top-level module. This doesn't directly help here currently, but what if exclusions in dependencies produced a warning of a possible incompatibility instead? I think that would solve the issue of you not knowing that an upgrade could break things.

I think a pretty good spot in the design space would be to respect specified incompatibilities with specific versions, but not ranges. Maybe even require a comment attached to the incompatibility constraint, which would explain the problem.

Bugs happen, and users should have some tools for that situation, but if a library is repeatedly releasing one broken version after another, you probably just shouldn't use it.

> but if a library is repeatedly releasing one broken version after another, you probably just shouldn't use it.

Your assumption here is that it was the boring-library version 1.5 that was broken, in reality it could be that they changed something that wasn't documented and was only an implementation detail, but was relied on by cool-framework (e.g. getting a list back sorted in one particular way in 1.4 and sorted in another way in 1.5 where the order was never part of the API contract).

Good point. In this case, the bug is in cool-framework, and I don't know that I see a great solution, regardless of the expressiveness of the constraint system. Ultimately I don't think there's a way to rule that out short of specifying exact versions. Unfortunately you can't retroactively change the version bounds for an existing release, so any library that specifies an upper bound admitting any yet-to-be-released version runs this risk.

Maximum bounds only help you if cool-framework developer knew about the incompatibility with boring-library when version 1.5.0 of cool-framework was released. If boring-library 1.5.0 was released after cool-framework 1.5.0 or the author of cool-framework didn't know you have a version of cool-framework that falsely claims to be compatible with boring-library 1.5.0 and whatever solver you use will be ok with that.

Pinning does not solve this. You still need a solver to generate new pin versions or update the pinned versions. Or do you manually pin/update each of the dependencies (including transitive ones)?

Yes: the transitive dependencies is the big problem here, and a hard one to solve.

I can handle direct dependencies manually, more or less, because that's the code I'm using directly and I know what are my requirements. But then I'm mostly clueless regarding dependencies of my dependencies, and if those aren't pinned by the project that is using them directly, all sort of bad things can happen.

And this is why I love Linux distributions so much: they've been solving this problem for a long time. Whenever I can, between using upstream or a slightly older version maintained by my distro, I choose the latter.

MVS sucks, except maybe to protect some users of libraries from their own stupidity. I'd rather not have to deal with that. The way Cargo and other tools do it works perfectly fine and is well integrated with SemVer. Default behavior is to upgrade to the latest non-major release, but it's easy to override for libraries that are more volatile or less trusted.

I'm amused...

While this was on the homepage or hacker news, vgo was marked as accepted. It's the timing I find amusing.

https://github.com/golang/go/issues/24301#issuecomment-39076...

Yes, it is amusing that this objection to the vgo proposal are only now coming forward when it was proposed 2018-03-20, 2 months ago, and the design document a month before then.

Russ and the core team ignored the dependency management issue for years, and only responded to the community-led `dep` initiative after a year and a half. The `vgo` proposal was something like a hundred pages of dense technical details, and Sam has a full-time job. I think the sense of urgency you're implying is both artificial and unfair. It's been years already; we can wait another month or two for the details to shake out.

>The `vgo` proposal was something like a hundred pages of dense technical details

Which document are you referring to? The one linked in https://github.com/golang/go/issues/24301 is only about 600 lines: https://github.com/golang/proposal/blob/master/design/24301-...

> The `vgo` proposal was something like a hundred pages of dense technical details, and Sam has a full-time job

Does that mean 2-3 months are not enough to read 100 pages document and respond specially from people who are most concerned about package management? Or that Sam should be the only one to respond to this?

Minimum-version-selection is exactly equivalent to giving an opaque cookie to the name of every version.

"I need libfoo version ScreamingOrangutan"

There's nothing to be done with a numeric string other than match it, so you might as well choose something memorable or with meaning to humans.

No? An MVS constraint like "1.2.0" will match any "1.X.Y" where X >= 2, and the one that's chosen is the first version that matches all the constraints.

Lets say you've walked the dependency graph for a project and somewhere dependency A is requested 3 times: "v1.0.0", "v1.2.0", "v1.2.1" which each mean (respectively):

    anything at or after v1.0.0
    anything at or after v1.2.0
    anything at or after v1.2.1

The minimum version that satisfies all of those constraints is simply the last listed version, v1.2.1, so that version is selected. Note, you can manually choose a later version by asking for it directly, e.g. add "A v1.2.10" directly to the dependencies of your root project and it will be chosen. Or add a new dependency that requests "A v1.3.0" or update a different dependency that requests "A v1.3.2" and the last one will be chosen.

I'm surprised there is still so much confusion about what MVS actually does. The actual algorithm is just a sort function. I blame the name, it doesn't describe what it does very well.

This is perfectly reasonable -- and not what was described to me when I asked about it, which also fit the name better than this does.

That is a terrible name.

Nope. MVS is designed so that each version can be explicitly ordered within a major version. Non versioned commits are assigned to the "v0" major version and compared by date of commit I believe.

Super confusing that they picked a name that's so similar to VG (the tool to manage your go workspaces, similar to python's virtualenv): https://github.com/getstream/vg

The name is not important, it's temporary. It's going to become the regular "go" tool in one of the next releases.

1. You should always fix your versions so things don't automatically upgrade 2. A good test suite will catch when things break so you don't find out in production. 3. Combining the above 2 it becomes easy to occasionally see which packages changed and upgrade to more recent versions. 4. After trying out a ton of dependency management tools (python, ruby, node, go, java) I think Yarn is the nicest solution i've worked with so far.

> don't automatically upgrade

You've missed the point; code neither automatically upgrades with MVS or with a lockfile. In both cases, it's a conscious decision to do so.

Helm likely had to update its dependencies (e.g. k8s dependency for k8s version compatibility) to solve a business problem, which makes it difficult to trivially downgrade that dependency (and thus break the business)

> A good test suite will catch when things break so you don't find out in production

helm has a good test suite; it wasn't good enough. A test suite is rarely, if ever, "good enough" to catch all bugs. That being the case, we must design in such a way that when a bug does occur, it can be easily resolved.

vgo reduces how many knobs we can twiddle to resolve a bug.

> After trying out a ton of dependency management tools (python, ruby, node, go, java)

None of those are dependency management tools. python has easy_install and two versions of pip. ruby has gem, bundler, and the mess that predated that. node has npm and yarn. go has dep, glide, gb, go get, and dozens of others. Java has maven, gradle, and others.

> I think Yarn is the nicest solution i've worked with so far

Totally irrelevant information without any additional data on that conclusion.

Reply on news.ycombinator.com