Comment by amasad | Hacker News Reader

Comment by amasad | Hacker News Reader

Parent Full thread

There are a lot of suboptimal design decisions that's shared across the state of the art in package management (cargo, pipenv, npm, yarn, etc) that the Go team is not taking for granted.

One is reproducible builds. The standard answer is lock files which is extra bloat and leads to merge conflicts. vgo is trying to avoid it with the "minimal versions" strategy (https://research.swtch.com/vgo-repro).

Another is shared (or recursive) dependencies. The semver answer to this is that if the versions match then they should be shared if not then they should be duplicated. But what do you do if it's a singleton, listens on a port, or exposes a port. Right now, with npm for example, good luck with that, you're in for a world of pain. On the other hand with vgo's "semantic import versioning" they're trying to make version interop more explicit.

Just because 90% of the time mainstream package mangers work doesn't mean it's a solved problem. Kudos for the Go team for trying to advance the state of the art.

The "minimal versions" strategy basically just replaces the lockfile with your dependency list.

Two branches want to update the same dependency? With a lockfile, they both update the lockfile and will have a merge conflict (assuming the updates aren't identical). With "minimal versions", they both update the file that declares the dependency, and, you guessed it, have a merge conflict.

Minimal versions put the burden of background compatibility on the maintainer of the dependency. If such is the case, library maintainers are constrained to refrain from breaking background compatibility at minor version changes, like Go itself does. In this case, you can merge, resolve the conflict by incrementing the dependency's version number, rebuild and retest and if all goes well commit. With lockfiles if branch X requires 1.4, branch Y requires 1.5 and 1.5 breaks 1.4, good luck with that. You'll need to update branch X to support 1.5.

SemVer already solves that problem.

Most package managers ignore the lock files of the dependencies when picking up versions - they only use the lockfile of the topmost project for picking up versions.

The big difference with the MVS strategy in vgo is that the dependency list of a dependency is actually used to determine the version. If package A uses B, which was tested with v1.2 of package C, you will get v1.2 of C, even if there is a later version of C available.

In the typical package manager scenario, the dependency list of B may be pointing to an old version of C which does not even work with B - there is generally no way to ask : give me the latest version of C which was tested with B.

I think package managers should have an option to run a build of your package using the oldest versions instead of newest versions, just to validate that the build works, but that this shouldn't be the normal way to resolve dependencies.

> The standard answer is lock files which is extra bloat and leads to merge conflicts.

Bloat how? A lockfile is a handful of bytes. A thousand lockfiles could fit in the space of a single Go hello world binary. And I've never heard of a merge conflict from a lockfile. What are the actual arguments against lockfiles?

More things, more state = bloat. Don't you agree that if you can do something without holding state it's invariably simpler? That's what MVS does.

There's also cases where your lock file and manifest are not in agreement. e.g. https://github.com/rust-lang/cargo/issues/4100

> Don't you agree that if you can do something without holding state it's invariably simpler?

Only if the two solutions under consideration are really solving the same problems. Sam Boyer alludes to this very thing in his discussion of MVS:

"If there are two algorithms that satisfy the same requirements, and only one is NP-complete, you pick the other one. That’s axiomatic. Moreover, if you have only an NP-complete algorithm for a particular problem, finding a less complex alternative that does the same job is an electrifying discovery. When such an alternative algorithm is proposed, however, the inevitable question to be answered is whether it actually does meet the original requirements. [...] But, in avoiding SAT, MVS also cuts out some of the complexities that I believe are essential to the domain. Being essential, the problems don’t go away when MVS ignores them. Instead, they’re redistributed into other, often less obvious places. If reading the vgo blog posts gave you a general sense of unease that you couldn’t put your finger on, that might’ve been you intuitively sensing some of these redistributions." https://sdboyer.io/vgo/intro/

There are situations where lock files, as implemented in some languages, can generate conflicts. Honestly I consider this a benefit, because I'd like to know about different expectations as early and explicitly as possible.

The bloat argument seems frankly absurd to me.

It's pretty easy to get a merge conflict from a lock file. If two developers add a dependency and then try to merge their changes together, it can happen easily.

In that scenario, you're going to get a merge conflict anyway in your manifest (e.g. in go.mod).

It's probably easier to manually merge the manifest (as they're much more human-readable) than a lock file though.

Lock files are easy enough to resolve that npm, in newer versions, will do it for you automatically, if you ask it to.

That's correct behavior though - you can't merge "upgrade A" and "upgrade B" in isolation with literally any confidence that it works.

>One is reproducible builds. The standard answer is lock files which is extra bloat and leads to merge conflicts.

You can’t have your cake and eat it too

Reply on news.ycombinator.com