Go’s Major Versioning Sucks – From a Fanboy
qvault.io
qvault.io
I disagree with much of this post.
> Go makes updating major versions so cumbersome that in the majority of cases, we have opted to just increment minor versions when we should increment major versions.
I disagree with this, wholeheartedly. I think Go's module system making modules separate packages on major versions is actually great once you actually know how it works. The problem is simply that it's non-obvious and not well understood. I think they could have made it more obvious requiring in on v1 for starters.
Beyond that you should absolutely be tagging major versions for every breaking change, and it should be something a developer has to stop and think about. Not something to be taken lightly, EVEN with regard to internal libraries.
You can learn how to code things preemptively so they are flexible for additive changes and require fewer breaking changes. It's almost always worth the effort in the long run.
To my posts point, the problem is just people don't know how it works, and it's design makes it something you learn late into a project, when you're tagging v2, rather than something you hit early.
FWIW, my post for anyone didn't catch it:
And that's fine, because when you break compatibility, you're actually not talking about the same library. And this makes upstream developers think twice about breaking compatibility. Accretion of features should be preferable to breakage.
It's what Rich Hickey talks about in his Spec-ulation talk: https://www.youtube.com/watch?v=oyLBGkS5ICk
I think most people, particularly those using Go, would agree with this.
It does not follow that we should make breaking changes easier or routine, nor that we should force people to use strict semver (which is not widely used for good reasons).
I see why they've done that as it simplifies assumptions but prefer the way other package managers handle this where it is left to producers and consumers to negotiate how strict they want to be.
Do you actually want that? Do you actually know that the multiple versions of the dependency you happen to be using do not conflict? If the dependency is in different namespaces their locks and other globals are also in different namespaces.
There are disadvantages to both and in particular duplicating dependencies should be seen as a short-term fix for a transition, not something you do routinely. If you do it routinely you'd see bugs due to shared locks, state, or changed data structures - it breaks assumptions about pkg globals for example, an important part of the language used extensively in the stdlib.
If you don't duplicate versions then you will have to wait for every single library in existence to update to the latest major version. This can take decades and still fail. Just take a look at Python 3.
Go's package management is really well designed I think and also actually semantically relying on semver
The problem is a mismatch between the Go team's perception of semver usage (or perhaps how they think it should be used) and its real-world usage. Some examples:
Major versions are used to communicate more than breaking changes.
Project identity doesn't change if a project changes, it is not a new package.
Hardly any projects use strict semver in the real world, and other package managers allow a more flexible, looser semver.
Just what is a breaking change anyway? Real-world packages make small security changes which break compatibility, sometimes minor changes do too, and that's ok if breakage is very limited in scope.
In the current go modules, new major versions are not easily discoverable or announced, and new importers will need to consult docs or know about go get @latest to find the latest version. This could be fixed with tooling if the problem is recognised. There are possible solutions if they want to keep /v2 paths, but we shouldn't pretend there are no problems caused by this change. It makes it easier to produce breaking changes and harder to version increment just for new features (as most projects from linux to rails do).
Personally I think it condones breaking changes and will lead to more breaking changes, not fewer - if you're forced to change import paths at higher versions why not just delete that api you don't like? Within Google say there are significant downsides to this and pressure to fix your own breakage, outside, not so much, including in packages people are required to use, like api client packages for popular services.
Significant breaking changes introduce real pain for consumers and in an open ecosystem should be avoided at all costs.
I think strict semver is reasonably possible for things that are libraries rather than products in their own right.
But you're right, people want major versions to indicate something significant and new (and occasionally major versions have a contractual significance as well - requiring customers to pay an upgrade fee), rather than small breaking changes. It's even sometimes possible to create a new library that has huge changes in the intended way you use it, its conceptual model and masses of new functionality, but it maintains backwards compatibility - it's perfectly understandable that the package owner wants to indicate this with a major version bump.
The other issue I've come across is that what constitutes a breaking change is much more subjective than many people realise. Any change is a breaking change if someone is reading in your library and tweaking bytes in specific locations. Of course, for most libraries they shouldn't be doing that, but that means that if someone is using your library wrong, then you don't worry about breaking them. But the question of if someone is using your library wrong is pretty subjective. At an extreme, that could be considered to be relying on any behaviour not explicitly documented as intended. Ultimately, it comes down to the judgement of the package maintainer, and that doesn't always match up with the judgement of the user.
Having version numbers that mean something to machines is very useful when it works though. Perhaps we should just separate those from human-targetted versions rather than go full sentimental versioning: http://sentimentalversioning.org/ Maybe something like <human version>.<machine readable api version>.<unique incrementing build number> could work.
Different producers/projects have different expectations for strictness in semver, and that's ok, they can use it differently in a negotiation with their consumers. Also different consumers have different requirements, and most tools provide a way for them to specify that on a per-import basis (upgrade only minor of these pls unattended, upgrade nothing on this one, only breaking changes on that one).
In short, loose semver is a feature, not a bug.
You would like to announce a new major version with fanfare, bumping a version number.
A programmer wants to be sure that his piece of software continues working if there has only been a minor version bump of a library he is using.
Did you read the reasoning of Russ Cox in respect to vgo, versioning and the module system in general? It reads very plausible and understandable.
Just because so many peopele interpret semver in their sense instead what it really means shouldn't prevent the Go core team to try to do it right.
There are multiple reasons given above why strict semantic versioning is not used in real-world software, that is a reductive summary of one of them.
Yes, I've read those articles a while back when they were written. In a very real sense, a consumer can never be sure that software keeps running properly if a library changes in any way without extensive checks. Real-world loose semantic versioning is a promise, not a proof, and it works better that way.
I don't personally think the current go proposal is terrible or sucks, but I do think it could be improved, and it'll be improved by listening to how people use versions.
It's just an API kind of thing. If a function / constant / var / type was available and is not anymore, it's a breaking change. It's an objective measure, there is nothing left to subjectivity.
Now, of course a bug fix is going to "break" things if your code relied on the bogus behavior. An updated algorithm can also break things (I maintain a SAT solver, whenever I update the underlying algorithm, even if it's way faster on average, in some limited cases it can make one very specific problem way slower to solve, which can have bad consequences). But as it has no consequence on the API part of things, it's not a major change in semver's meaning.
https://www.bennadel.com/blog/3501-when-is-a-change-a-breaki...
Or are they? Check this out: https://www.unisonweb.org/docs/tour#-the-big-technical-idea
Strict semver (as proposed by go mod) - any breaking change means a new major version, and a 'new package' at a new import url. This means older users are left behind unless they explicitly upgrade and producers are encouraged to make breaking changes because it is easy for them. At present there are no measures to alert consumers to upgrade or communicate what the changes are. This doesn't reflect current practice even in the Go project itself (at 1.x despite small breaking changes/deprecations).
Loose semver (as used in the real world) - package identity is constant, minor breakage happens at every patch level and the level of breakage is negotiated between producers and consumers at the package level. semver is used to signal changes (major - big changes, possible breakage; minor - small changes, less breakage; patch - tiny changes, possible breakage to that area). Note major versions usually are used for big changes, not just breaking changes and big changes can be just as painful for consumers (e.g. in v3 we're introducing a new api for payments, start using it, the old one is still there but will go away soon in v3.1). Usually package management systems provide mechanisms to help that negotiation (automatic upgrades within minor/patch on the assumption breakage is minimal).
There are good reasons for the current go mod defaults (simple dependency resolution, simple slow migration between versions), but they do ignore real-world usage IMO and will lead to a bit of pain without some further work.
A convenient tip: in the real world this is almost never true, so when you find yourself insisting that it is, stop and think.
Go the language is really best in class when it comes to keeping backwards compatibility.
And at this point, I just stopped digging down that rabbit hole and instead added
TESTIFY := github.com/stretchr/testify
mkdir -p "$(GOPATH)/src/$(TESTIFY)" ; git clone --depth 1 --branch v1.3.0 https://$(TESTIFY).git "$(GOPATH)/src/$(TESTIFY)"
Isn't the one of the reasons we use semver in the first place so that after you do some small change to the current application you're working on, you don't suddenly find yourself having to update 2/3 of your the environment just to compile it?
If they used go mod and didn't see it as a breaking change you'd be in the same situation. I think many people would debate whether this is breaking or not or deserves a major version - I think most people would say no if that was the only change.
If they used go mod and did see this as a breaking change, they'd increment the major version and you'd never notice. The unfortunate consequence is most of their consumers would continue happily using a version 1.3 multiple major versions behind and missing critical security fixes (most projects backport fixes for say two major versions) - and they'd probably end up with users on a range of major versions from 1.x to 21.x if this is your definition of a significant breaking change.
In theory under strong semver every single possible breaking change would be a new major version and every bit of software more than a few years old would be on version 945.5.1 - in practice, in the real world we never see that sort of versioning, and people use major versions for something very different (for significant changes in api surface (whether breaking or not), and sometimes for significant breaking changes). It's a signal rather than a hard objective rule.
None of this is insurmountable of course, but it does need to be attended to. At present go mod seems to assume strong semver and the result would be IMO a proliferation of breaking changes and outdated software being used (as opposed to current Go which almost forces everyone to be on HEAD and to try to avoid breaking others).
I don't argue for strict semver, you can never be sure if a supposedly minor change won't actually break things, sure, but some changes are guaranteed to break things. Why not at least mark them with the version bump?
What would be ideal is if we had a means of communicating more information about releases of code and their relationship than semver provides. I.e. I can publish a repo and add some kind of "language-support.json" document that specifies I support the 3 latest major releases of Go, and have the package manager figure out whether my version of Go is supported. Other ideas for metadata would be the ability to add labels to releases, and have the option to filter/prioritize upgrades based on those.
I would love if package managers supported labels as part of the metadata, and I could get a summary of all the labels between my current version and the new version. So on an upgrade, I could get a label diff for the package versions like "security:cve-1234 feature:oauth2-support bugfix:stale-kafka-messages". Those are cherry picked things that make nice labels, not everything makes sense in those messages. But sometimes it feels like we just do global updates and make sure everything builds, just to keep the tech debt low. I have no idea what actually changed in the package, and as long as it builds and tests pass I don't have to know. That's because we have so many dependencies, and it would take ages to read the release notes for all of the versions of all of the dependencies between our current version and the new one. Labels provide a means to communicate a succinct version of what changed; succinct enough for someone to read while they're waiting on tests to run.
I'm pretty sure a huge number of Go projects no longer work with earlier Go versions, it's just a question of how far back you go. If you're back at Go 1.10 I'd recommend freezing your dependencies and putting them in your own repo, it's the only way to be sure. Otherwise this is likely to happen to you again as people aren't great about supporting all versions of Go 1.x - it's quite a support burden.
I would note that even the Go language itself has dropped architectures and support for bootstrapping with older Go versions without bumping the major version. I think that's fine. Of course that does break strict semver (oops), but nobody cares because nobody actually expects strict semver in real software. What's important is the impact - it's sometimes ok to remove features if nobody is using them and I've never seen a project use a major version for that. I've also never seen a project bump a major version for a breaking bug fix.
It is a new package. From a practical perspective once you break backwards compatibility the new major version is its own independent thing. If you pretend its the same thing then you run into the situation where two libraries need v1 and v2 and therefore can't coexist. This is going to destroy your entire ecosystem. Replacing a few strings is nothing in comparison.
Not saying its right or wrong, but clearly it is not a death sentence.
Linux distros have the same issue, rpm or .deb packages cannot have the same name but still support having 2 versions of a package/library installed.
If it's a shared library and they don't get the SONAME versioning right for incompatible backwards compabiliry, it's even more cumbersome, and incurs more work for everyone to consume that library.
That perspective isn't appropriate to use outside of a narrow scope. In particular it's not appropriate to use at an ecosystem level, it's too surprising.
Part of the appeal of GO is simplicity: get some things down in a minimal yet complete way then consistently do it on the strength that it's a net-win: less complexity to learn that takes you further in most cases. This isn't the only game in town, but it is a good game. C++ is different: we've got 21 million ways of doing things, but you only pay for what you use. That's fine too if you know what you're doing.
Back to GO: The fact that the real world is wishy-washy here sometimes in the mood, sometimes not, sometimes I like blondes some days I like brunettes other days isn't the go way ... is a sometimes depressing part of DEVOPS: not formal. It's not the GO way either.
So I think GO's standpoint that changing something to explicitly ask for a breaking change isn't out of line. It can't be that tough of an ask to mean what you say and say what you mean when we label code with a semantic version.
Both this post and the other V2 problem GO post fail to better get at the crux of the problem: In GO there are two names:
- the path we import in code
- the actual GIT URL (or vendor ref) down to tag/commitId in go.mod that the previous item ultimately resolves to
The module name in code is ambiguous. It's desirable to some that code is unchanged ever wrt to imported module name. If true, then we must turn our attention to what's in go.mod. Here the ask is for hints that there's a breaking version available ... and it'll be up to the DEV to use it or ignore it changing go.mod as they deem makes sense.
If false, then you've gotta change the code to reference the breaking version if that's the desire ... realizing it's still ambiguous because it doesn't resolve down to a commitId or tag ... so that leaves go.mod on the table for change too.
Nobody debates that code must change to reflect breaking changes if the breaking change is included in go.mod. The questions are: did we ask? Where? Did we know there's a breaking change, and can't we get some help knowing there's a breaking change since GO hits GIT anyway?
At it's crux then ... there are two names. Leaving the code unchanged and changing go.mod with some tool help is better IMHO.
If people aren't using it, it's weird.
That smells of trouble to me... either it should be good enough people want to use it, or painful enough not using it that people grit their teeth and do the right thing because its less effort in the long term (eg. you cannot use module at all unless you do it right).
If there is no obvious downside to doing the 'wrong' thing, and it's less effort, and people are doing the 'wrong thing', in practice, in the wild...
...well, I'm not sure this has really worked out ideally.
Certainly, "great" is not how I would describe it.
That said, Go team should have made it clear right in the title (for all blog posts) that they are attempting a unique non-intuitive solution to a hard problem of transition from 'no-versioned' world of existing packages to 'semver' and the explicit desire for force people to avoid versioning if they can (just like the old days). There is nothing wrong with weird solutions. We don't have to follow Rust or Ruby or npm. Their situation is different.
Of course then the protobuf or grpc team (I forget which one) went and created a new "v2" without the "v2" suffix and that pissed a few people off. Of course that was still within the reasoning Russ explained (it was a reimplementation and domain part of the URL already changed, so there was no need to add v2), and Russ probably doesn't control the particular package's team. But that is life! People still expect perfection from opensource projects.
However if I use semver internally, like it's supposed to and make regular major changes (let's say once a month), my code repo becomes a mess. After a 2 years, I'll have folders for v1 up to v24. And the fact that I have to copy -R/grep|sed, means that people will be less likely to properly signal that a backwards incompatible change has occured.
Disregarding uniformity, for the reasons above I don't think their design is a good one. It treats major version changes as a rare event.
For a collection of code that someone calls a library, all bets are off.
This will just pollute the API interface for the sake of maintaining backwards compatibility.
If you properly honor semver, for any non-static library, major version changes will either be a modestly-frequent event, or you're really straightjacketing yourself with your first v1, or you're a genius who got everything right the first time and never have to make any changes.
Or, most likely, you're not honoring semver properly and kindasorta squint at your changes until they don't seem important anymore and then ship an incremental anyhow.
Go may make it somewhat more annoying than the average language to ship a new major version in some ways, but libraries either not rev'ing their major version number when they technically should have and mostly expecting to mostly get away with it, or libraries bending themselves slowly into pretzels to stay on the same major version according to semver when they really ought to just make a new major version are things I see in many languages.
IMO, someone tackling a problem for the first time shouldn't be writing public libraries. At the very least, people shouldn't be using it.
Quality, stable libraries tend to be written by people who already have at least 1 or 2 iterations under their belt. Maybe not complete iterations, and rarely public, but enough experience with the nuts & bolts of the problem, including the problem of balancing concerns between caller & callee, to have a good idea about what the proper interfaces should look like to ensure long-term stability. If someone doesn't have a strong sense that they could support the v1 interface in perpetuity, then maybe don't create a separate project. Not only does it save everybody else from the pain, it often saves the author the pain of constant churn, presuming the author even uses it in more than a handful of their own projects.
For many of my own open source projects, I often have years of abandoned false starts lying around, or various just-so solutions manually copied and hacked into various other projects. Only when I feel like things have finally congealed do I then bother to write a dedicated library, whether private or public, and migrate everything to it. One way to know things are looking good is when you realize you're no longer fiddling with the API, or maybe just vacillating between two equally decent alternatives.
In my head - that's what v0 is for. Many people try to jump the v1 gun well before they've allowed the real world to impose itself on the unwarranted assumptions that should have been shaken out in v0
"For a well architect house, fires should occur very rarely. As such we should not install smoke alarms."
If it can happen it will happen and whatever you're designing should be built with that in mind. Say nothing of an entire language's package management semantics
When the surface you're looking at is "all open source code supporting this language" are you confident saying that there will only be 1 breaking change every 2 years for your tools?
Imagine you're making a GUI. I don't think there's any GUI lib in the mobile OS space that hasn't added or deprecated a major feature in the last 2 years. Android is very good at this and there's still large deprecation events (see Jetpack).
No breaking changes.
The HTML-based applications I've written going back the last six years all still work on modern browsers.
The few apps I've written for Android didn't work across all Android versions even when in active development. Android really is the best example of how miserable development can be when backwards compatibility (or really, just compatibility) isn't taken seriously.
Tell people you have an unstable library. Breaking changes _will_ happen. Then, when you've used your library in production for a while and are reasonably sure it wont have to change, tell people it's stable. If you make breaking changes after that, then start with a v2 suffix.
For some reason, semver has made people stop going through the alpha, beta and rc stages of a release, even though the point of such things is to discover breaking changes before you're commited to maintaining the version for the time to come.
Major version changes should be a rare event. If it's not, you're really just shipping around a (pre)alpha library, and should be explicit about that.
The whole software industry moved away from waterfall to agile because it's unreasonable to assume that no requirements will emerge. I would disagree, for example, that Python3 is not ready just because they're introduced backward incompatible changes in some minor releases.
<grin>
Regarding Python, I no longer see 2.7 in the wild (I have no doubt it is still in some places) all projects I'm working on are in Python 3. AWS Lambda supports 3.8 (which is the latest stable version), Ubuntu comes with 3 installed by default. FreeBSD set 3.7 as default, depreciated and will remove 2.7 completely by the end of this year.
Many Python 3 packages were released as python 3 only, the packages that that previously supported 2 dropped their support. Currently running 2.7 is a liability. Not as much that it is EOL and there are no security fixes anymore, but also packages dropping support (which is IMO a bigger deal).
If anything, Python is chastised for having maintained two major versions in parallel without giving any compelling reason to force major rewrites on the while ecosystem.
That's completely unrelated to library versioning. The software industry that I work at absolutely relies on library authors to diligently introduce breaking changes only in rare occasions, and preferably using semver to the extent possible.
If you can't guarantee your library will be kept up-to-date with bug and security fixes, without me having to re-write my application on minor library updates because you decided to change your API to make it prettier, then I most certainly won't be using your library (indeed, I have removed dependencies from projects I worked on for this reason).
This is the tension of semver - a minor library update might include a breaking change, but that doesn't mean you have to rewrite your application. For example, lets say I have a library that returned an int32 in a single function that would silently corrupt your data, but now has to return an int64.
The right thing to do would be to issue a major upgrade where the function returns an int64. If you never used this function you wouldn't have to rewrite your application. Major updates in semver only mean backwards compatibility is broken, it doesn't mean it's am whole new api.
To me, the Go approach takes your approach to major upgrades - reserve them for when you do something like rewriting the API. That's to me isn't a good assumption to bake in, I'd like to change a single function signature without having to copy my entire library to a new folder.
Then make a new function within the same folder that works in a new way. If necessary, mark the old function as deprecated. An exception can be made if the function never worked to begin with, but that should really have been cought by tests or people using a pre-release version of the library.
It really is not. The initial assertion was that stumbling on breaking changes once per month was somehow a sign that a library isn't ready. Thus assertion flies in the face of the reality of software engineering. If your whole process is designed to allow requirements to emerge then you will have to deal with emergent requirements that you need to accommodate, and in moderately sized projects that always involves changing behavior.
You seem to be assuming we use libraries to implement the quickly changing business requirements? If you do that, then sure, you'll have a hard time using versioned libraries at all. But you don't need to do that. Doing that is just a pain.
Have a single code base with separate modules by all means, but don't pretend the modules are versioned separately and independent.
In other words: don't make them libraries unless you MUST. A library, by definition, is a re-usable software component. Something can only be re-usable when it's stable (if it keeps changing, it's by definition not re-usable as consumers would also be expected to keep changing, and that's not the case for everything as you seem to assume). Things like base64, encryption algorithms, data structures... even spec-backed things like OAuth and HTTP libraries don't just change every month.
There's also an insidious side effect that people tend to think of the more common semver of features than strict semver, and they associate versions with progress and effort. You could easily get to version 50.0.0 just sorting out finishing touches on the API with strict semver if you don't have good communication with the customer. Some enterprising middle manager will start asking you've spent 50 major versions worth of effort on this library or app.
As a result of those effects and the unequal application of semver, I usually encourage people at work to choose other versioning schemes. Without strictly enforcing semver, there's really not a huge advantage to semver over a plain incrementing version. Incrementing versions also have the nifty feature of making it trivial to see how many versions behind you are. Or some people go with timestamped versions, which tells you how old your version is.
I think the Go module system encourages you to avoid adding unnecessary boundaries. As someone else pointed out already, one of the problems is that you don't find out that boundaries are bad until you hit v2, and then you discover that you have this chunk of technical debt (unnecessary repo boundaries that slow down development) that you didn't know about.
Imagine spending an significant portion of your day migrating all your internal apps to the latest major version of an internal package, then you can imagine what we faced once a month.
We stopped doing breaking changes after a while. Instead we're effectively doing versioning at the module level. If module A could really use a fixup, we make an entirely new module A.V2, and start using that in new code. Things that don't need the new functionality can keep using the old API, which still works, and upgrade whenever we have some extra time (we never have extra time) or we really _really_ need to.
We've also adopted a policy that nothing gets added to our internal packages before they've been used in an application for a certain amount of time. Code duplication between applications is fine if we don't know exactly how the API should work, that way you have flexibility to change the code to fit a specific use case for a specific application.
If that common functionality really has to be common _straight_ away (which we surprisingly haven't hit yet) we could place it in a module A.Unstable to make certain everyone knows that it can break at any point.
In either case. It's just as important to avoid breaking changes in internal software as in everywhere else. Breaking changes are not free, except for the library authors.
That's one of the points of confusion — first that the go.mod module declaration doesn't need to exactly match the module location, and secondly, following this, that using a folder instead is a fallback option.
Go is famous for favouring simplicity in its design, but this is a rare instance where it's not made things simple enough.
If you make it hard for tools like Snyk to work with your language I'm not going to recommend it to anyone.
I actually like Golang in specific circumstances (backend simple micro services, anything docker, and utility binaries) but they picked the versioning scheme as the, "This is the dumb hill we're going to die on," that every language has in some fashion (ex: no case statements in python, perl 6 coming soon, it's 2001 and we like it that way for php, java... just, all of it).
They are paying for, say 100 engineers (just making up this number). Isn't that enough?
I don't want to go as far as argumentum ad antiquitatem but if you break KISS and the principle of least astonishment it should probably be worth the hassle.
>Their situation is different.
Is it really, though?
The original Go protobuf implementation is in the module "github.com/golang/protobuf".
A couple years back, we decided that it was time to do a major overhaul of the protobuf implementation. Protobufs were one of the first major pieces of Go code outside the standard library (the second post ever on the Go blog is about it), and there were a number of design decisions that made sense at the time but which we wanted to go back and revisit. Doing all of this was not going to be possible while preserving API compatibility.
Given that, we needed a new module path. That's the core of Go modules and API changes: You change the module path when you make an incompatible change. You can do this by bumping the major version of the module (since the module path includes the version), or by changing the module path entirely. (There's another option, which I think doesn't get enough attention: You can just make the change, and force your users to deal with it. If you're okay with breaking your users when they upgrade, nothing stops you from doing this. We take backwards compatibility very seriously, so that wasn't an option for us.)
Given that we needed a new module path anyway, we took advantage of the opportunity to move to a path that isn't specific to a particular code hosting provider: "google.golang.org/protobuf". GitHub is great, we love GitHib, but we'd much rather have a provider-agnostic name for the module.
So we have a new module path. The question then was: What version do we tag the new module as when we release it? Remember that there has never been any version of "google.golang.org/protobuf" at this point; it's a completely new module.
One option was to tag it as v1.0.0. But that might be confusing, because this is "version 2" of the protobuf implementation, even if it's under a different module path. Also, if someone complains about a problem with "v1.2.0" and doesn't mention which module they're using, we won't have any way to tell which one they're talking about.
Another option was to tag it as v2.0.0. But that might be confusing, because there's never been a v1 of this package. Also, this would likely cause unnecessary confusion with "proto2" and "proto3" (versions of the protocol buffer language).
We waffled for quite a while, and eventually settled on v1.20.0, which avoids the missing v1 weirdness but also ensures that there is no overlap in versions between the two modules.
That confused people. Alas. Perhaps every option would have led to confusion.
In retrospect, perhaps we should have gone with v2.0.0 (and a module name of "google.golang.org/protobuf/v2"). Too late now; oh, well. If this was the worst decision we made in the new implementation (and so far, it looks like it may have been), I'll be quite happy.
I think this post is an example of a package not adhereing to its principle; a major revision implies breaking changes, so it doesn't make sense to have a tool "automatically update" things for you, because theres no garuntee that it'll work.
I feel like the issue is that this post doesn't totally understand how the package versioning is meant to work in the first place; if you're changing your struct names and it has no external effects, i don't see why you /want/ to semver, because you're exactly getting much value from semver'ing other than the vanity of pretty numbers. Maybe I'm missing something?
Im curious if an ideal package system that would work for this particular project, but I don't think one does.
The whole point of semver is to address the issue of backward-incompatible changes breaking downstream dependencies (well, among important other things).
What Go is doing is forcing you to use Semver the way it is intended to be used; from the semver specification:
---
Given a version number MAJOR.MINOR.PATCH, increment the:
MAJOR version when you make incompatible API changes,
MINOR version when you add functionality in a backwards compatible manner, and
PATCH version when you make backwards compatible bug fixes.
---
If you don't want to adhere to this, just cut PATCH versions and MINOR versions.
It's an absolute terrible idea though.
As annoying as it may seem, there's a lot of benefits to using semver the way it is intended to: it forces you, the developer, to think a little more about how you design your code.
The small amount of productivity you lose getting used to do the proper thing (i.e. cut MAJOR version, or spend more time designing your code so that it more forward thinking) will pay off in the long term.
The tiny amount of productivity you gain by not doing the right thing will come back to haunt you and make you wish you had been more rigorous.
His startup might be small right now, but if it becomes successful, their number of internal users might explode overnight and they'll suddenly wish they had used semver properly all along.
When you are crunching for a deadline, the last thing you want is "oops the tool automatically bumped all the major versions on you" when you didn't explicitly intend to. Having a dedicated flag would be nicer than grep/sed though.
I want to get to a v3.x.y but I need. a bunch of pre-3 major versions first. Linux kernel used to do with odd numbers for experimental versions and even numbers for stable versions.
I am not a Go user, but if it is truly the case that you get no notification when a new major version is available, that could be annoying.
Only if the older major version is deprecated. I do not want to be annoyed with the existence of new major versions unless I have to. My code already works with the older version.
Its trivial to bump versions in countless package managers but its frustrating in Go. Telling this dev that he's holding it wrong seems like undue push back. Why should this be cumbersome? I don't think accidentally taking major versions is a problem in most package managers.
Of course, you could handle this w/ more complexity in the package manager/build system, but having separate import paths makes it very clear, and is a very "Go"ish solution.
Since you're only doing this when you've got breaking changes coming in anyway, i.e., you've already got to fix the code, is it really such a big deal to update the import paths while you're at it? It's like a 2 second grep/replace...
That said, having Go tell you about major version upgrades available, and also an option to fix the code to use the upgraded versions import path, are good ideas.
In theory this sounds good but in practice major versions can be drastically incompatible, and I wouldn't expect this to work very well at all in practice.
One of the excellent features Go's use of semantic versioning enables is for lower major versions to be updated to use the higher major version implementations, as a sort of backwards-compatible shim. In this case, no tools needed.
Just because the module broke compat doesn't mean you were using the things that broke though. Maybe they dropped or changed functions that you didn't use.
But you're right, in some cases the breaking changes won't break you. In any case, I do think it's worth having a dead-easy way to fix the imports, rather than having to fuss about with grep.
(And what if you could actually ship rules to transform code to the upgraded version? That would be cool...)
Writing about a complaint in a blog post and saying Tweet me is a tacit admission that the author isn't willing to face criticism from experts and just wants to get attention from the casual unininformed masses.
I'm not a Go user at all, but it reminds me a bit of like back when Gnome3 took a direction completely different than what I loved about Gnome2 (Apple worship, largely), which eventually lead to projects like MATE and Cinnamon.
You don't have to defer completely to the authority. But if you want to be treated as one, you should be willing to engage and consult with them.
If that is the case do you have to follow rules for semver? (edit- that is indeed what they do) Your users are all internal, its not like they are going to rat you out to the police for agreeing to do something different internally. It's just a couple of numbers...
I don't do go but, as described, the version system seems fair and reasonable on paper, for a system designed to avoid suddenly breaking code that depends on it.
The Go module system is a debacle. It's not intuitive and it's badly broken right now.
Not making apologetics here, I just don’t know what a good package system looks like.
- Haskell: Stack with stackage (mainly because of stackage)
- Rust: Cargo and crates
They are both above and beyond the rest of the space.
I'm not convinced that the stackage approach is scalable to larger ecosystems... Haskell has a fraction of Go's popularity.
Say what you want about Maven the build system (I’m not going to die on that hill, even though I also prefer Maven to virtually every other build tool I’ve worked with) - but Maven Central and Maven the Package Manager are solid.
I've been working on an android app and use gradle with the kotlin dsl. I haven't really done any programming with the build steps but this is pretty sexy and doesn't need three open/close tags every plugin.
dependencies {
implementation(fileTree(mapOf("dir" to "libs", "include" to listOf("*.jar"))))
implementation("com.google.android.material:material:1.2.0")
The problem with Gradle is that it's flexibility is... dangerous and without some constraint can quickly lead to a lot of pain.
JAPH. :)
it’s pretty sad really if you think about it. that there are legitimate use cases where go is really really well suited for the problem.
my vote is on someone coming up with a sane approach at some point and people slowly adopting it.
From all the mgmt I had to use, Python is by far the worst.
Every time I have something in Python to use I pray that pip will install it correctly which is roughly 50% of the cases.
also, there is a difference between installing packages and environment management
If you want to break your existing code, import a different package.
If Haskell followed this principle, cabal hell wouldn't exist. If Windows followed this system, DLL hell wouldn't exist.
looks at watch
20 years ago.
Makes you wonder about the go team, like where were they exactly between 1985 and 2005?
Of course, if this increased the number of forks, it would increase the times we need to convert between types provided by "different" libraries - which may or may not be a problem, but isn't "a Cabal problem".
How it would actually work out in practice would depend a lot on how it shaped behavior; it sounds like srtjstjsj believes it went well for go, and I don't know enough to weigh in there in either direction.
I just wanted to point out that the big problems with dependencies in the Haskell ecosystem, sometimes called "cabal hell", have not existed for at least the last few years (since the introduction of v2-style builds).
Exactly!
Ross Cox explains the concept in this talk from 9:20: https://youtu.be/F8nrpe0XWRg?t=560
I regularly update across major versions of packages in other languages and don't touch a thing other than the package version. I appreciate the version bump so I can do my due diligence to research the changes, but I just don't use the entire API of libraries in most cases so I'm not always affected by changes.
Perhaps I’m yelling at clouds, but it’s disappointing to find an article that opens with “fanboy” and closes with “suck it” making a debut on the first page of HN.
The best option for projects is probably to loudly declare that you are not using semantic versioning, and then use it anyway, on the good-faith basis that is the only actually possible way to use it. Declaring that you don't use it will head off trouble from people who expect it to be an impossible magic constraint, while using it will help people make sense of your version numbers.
* we're using projects internally and don't want the hassle of rewriting import paths when making backwards-incompatible changes
* we don't want to just give up on semver and publish breaking changing on the v0/1 numbering scheme
I feel like you have to pick one. If it's the first, then it's actually a positive thing that explicit import path rewrites are needed in my mind at least.
> new copy of the entire codebase
What i have done in my module, and what i think is a common way -- is to move common codebase to common submodule, which will be used by all other versions.
> Allow package maintainers to specify the major version simply by updating git tags
Wouldn't it cause problems if developers would need to maintain multiple versions in parallel?
> command has no way to automatically update to a new major version
To be honest, i don't see why there should be auto update to new major versions, since, usually new major version means breaking changes to API.
go's not alone in breaking changes between minor versions, and "what worked yesterday no longer works" problems. v8 is very similar, where a very large breaking change can occur from one minor version to another.
even without a loose "agreement" like semver, it feels like there is no thought to "outside the bubble".
Sorry, this is incorrect: https://golang.org/ref/mod#module-path. A module path describes where to find it, starting with its _repository root_ (github.com, golang.org, etc) and then the subdirectory in the repository that the module is defined in (if not the root).
So, if a module lives in golang.org/username/reponame/go.mod, its module path is likely golang.org/username/reponame. If a module lives in golang.org/username/reponame/dirname/go.mod, its module path is likely golang.org/username/reponame/dirname. (and so on with a /v2 folder, etc)
I mention this because it appears that OP's major gripe in "package side problems" is that the /v2 dir "breaks" the (mis)conception that a module path describes _only_ the repo root.
(see also: multi module repositories)
> In other words, we should only increment major versions when making breaking changes
No, you can increment major versions whenever you want (though it's painful to your users). But, you _should_ increment a major version when you make a breaking change.
> I think a simple console warning would have been a better solution than forcing a cumbersome updating strategy on the community.
Could you elaborate on how a console warning solves the problem of users becoming broken when module authors make incompatible changes within a major version?
> Another problem on the client-side is that we don’t only need to update go.mod, but we actually need to grep through our codebase and change each import statement to point to the new major version:
What if you need to use v2 and v4 of golang.org/foo/bar? How would you import them both without one having a /v2 suffix and a /v4 suffix? Are you proposing that users should only be able to use one major version of a library at a time?
(I assume you are talking about a user upgrading to a new major, not the package author bumping to a new major. If the latter, a grep and replace is quite approachable and is shown in the blog you linked :) )
> Go makes updating major versions so cumbersome that in the majority of cases, we have opted to just increment minor versions when we should increment major versions.
I'm reminded of when the "unused imports not allowed" rule was lambasted, and then goimports was released and the conversation was snuffed out. This situation feels analagous.
You praise the toolchain and the good decisions in modules, but then hinge your thesis against it on "it's cumbersome". That's a valid concern, but it's likely that a tool that makes major version upgrades easier will resolve your issue. A wholly different design certainly is not needed .
Check out https://godoc.org/golang.org/x/exp/cmd/gorelease for one tool that's under development aimed at helping version bumps. It sounds like you also need something that will create the v2 branch/directory, change all the self-referencing import paths in that branch/directory, and change the go.mod path. That sounds like an easy tool to write - I expect something like that should come out quite quickly if it doesn't already exist.
-------------------
Side note: In my opinion, dependency management is a rat's nest of choices that seem good at the outset and end up with terrible consequences later on. Go modules make super well thought out decisions, and is a very very simple design, built to last for a long time without regrets. Sometimes the right answer is to work around a small problem with some tooling or documentation rather than go a totally different direction that will have large, sad consequences later.
That is: all choices have downsides, but it's good to choose the best choice whose downsides can largely be tackled with easy solutions like tools and docs. Decisions like "we should build a SAT solver" have sad, sad, sad consequences that can't be tool'd and doc'd away, for example.
Welcome to the Disillusioned Fanboy Club, there's a bunch of Perl guys who eventually gave up and moved to Python waiting at the bar with some war stories for you... (I'm the one alternating beers and bourbons...)
Why does that have to change if you move from v1 to v2?
OP seems to have a history of repeatedly spamming their company's posts https://news.ycombinator.com/submitted?id=lanecwagner
If there is an article I wrote that I think will do well I usually give it 2 or 3 chances