Fix the versioning
github.com
github.com
I'm a big fan of semver, but I also test whenever I do a rm Gemfile.lock; bundle install (I've only managed a fairly small nodejs app in production, but I'd be surprised if people really don't test something like that). So he did something we think is bad -- but plenty of beginning engineers do this all the time, and while the discussion on github is appropriate, I'm not sure it belongs here on HN. People shouldn't be afraid of making something useful (as underscore is) because of the potential for extremely public shaming.
If I'm the author of an unpopular module, where I am the sole user, and I don't like semantic versioning or some other facet of package management, I don't follow it. Should I be expected to change my stance just because other people begin using my project? At what point is my project's community large enough that maintaining a healthy ecosystem involves sharing some control over the practices with that community?
It also doesn't help here that rather than having a rational discussion, the author released what can only be described as a troll package, where every version change becomes a major version. Using that would be the height of insanity, as noted in the issue thread, because all libraries would need to update their dependencies and re-release with every Underscore-semver release, due to dependency version conflicts.
What exactly about Jeremy Ashkenas makes you think this was done as a way to troll people? He still took a lot of time to make release notes and note all of the changes.
He's not released a version that obeys semver, he's released a version that treats any update as a breaking change.
> He's not released a version that obeys semver, he's released a version that treats any update as a breaking change.
SemVer isn't a law. It's a set of guidelines for versioning software in a slightly less arbitrary way. Maybe extra precaution could be taken for something as common as underscore, and to me that is the only argument worth having here.
That said, I do write code and have tools that are used by other groups of people, and have experienced feature requests and suggestions and patches that I don't necessarily agree with, and I've always tried to step back and consider the overall impact on the project rather than my personal preconception of what the project "should" be. I've merged features that I'd never use and adjusted things to make my project more friendly for users who aren't me.
I think that one of the greater things about open source projects is that over time the original author can focus more on the fun parts or on guiding the overall direction of the project, because the community can provide assistance with identifying issues and maintaining existing systems. Whether the version number is 2.0.0 or 1.7.0 or 50.0.0 is such a trivial thing, and as a developer I'd imagine I'd make my choice based on what made my project more usable for the community.
I think it's OK for an open-source project – a freely-offered gift to the world – to take that approach, if the author prefers to be personally indifferent about "breaking changes". It's a sort of technical declaration of "NO WARRANTIES", specifically no warranty of "FITNESS FOR A PARTICULAR PURPOSE" – that purpose being, "painless drop-in where a previous version was used".
This is the reason that semver and ranges of dependency management became useful in the first place, because I can say that leetapp needs underscore-realsemver (1.6.0-1.6.n), and tablething can say it needs version (1.6.3-1.6.n) and so forth. This works with sane semver because we can use the parts of the version string to agree on the major API while allowing bugfixes and other improvements to happen without every project needing to bump version numbers.
Historically, nearly every single release has had at least one thing that mildly changed the behavior of some minor edge case or other. So by strict SemVer, if we wouldn't be at version "170.0.0" by now, we'd be damn close to it.
I knew this was going to happen when NPM went from default ~ in dependencies (update only patches) to ^ (update minor versions). In fact, I changed my default back to ~ out of fear this would happen.
edit: Please comment saying why you disagree, rather than simply downvoting.
Respectfully disagree on the particulars, sure, but don't imply that someone who's given you a gift now owes you extra work, when their own design sense – the same thing that created the shared bounty – guides them otherwise.
People should also consider that authors of popular projects are inundated with support requests and demands for their time, and have to set some boundaries on their efforts, which may appear callous to outsiders. If his boundary here is not wanting to embrace Semantic Versioning, respect it, and fork/migrate/workaround rather than whine/shame/guilt.
But if your point is that we should publicly highlight mistakes (or things we don't like) in packages because we expect people to do better... then I guess I should step out of this conversation. I don't want to talk highschool about someone building cool things.
As a developer, how you treat your user base is relevant, and has a real impact on the value of the thing you have produced. We have successful projects whose leaders vary wildly in style, from Linus's approach to kernel development to projects that delegate final code control to community contributors.
Talking about the human elements that factor in to technology has merit; talking about the best ways to handle a project's community has merit. It's not "talk[ing] highschool".
_ is the most depended on node.js module. Maintainer, Ashkenas, introduced breaking changes. By default npm will pull down these changes without prompting because it was designed to follow semver. Maybe npm is at fault. Maybe it's Ashkenas.
However, Ashkenas knows his change is going to break people and just doesn't care.
You can suggested that every package on npm vender their own dependencies, but in practice nobody does this because npm is known to make these assumptions.
This is the most depended on module on npm. Why are you so intent on alienating the package consumers? IMO, having a version 47.0.0 and such doesn't seem all that terrible to me, and is still just as human readable. Other groups are doing it and it's working just fine (ie Chrome).
What would the effect of that be? Since few people are silly enough to use the ">=" operator in package.json, a popular package with that many "breaking" changes is effectively going to be pinned at 46 different points. I imagine we'd start to see some of the same entitled whinging we see exhibited here, but in the opposite direction. "Why do you have so many major releases? It means I have to re-release my packages constantly."
2) Breaking semver sucks, and I go hulk rage mad when my shit breaks.
Semver should be based upon the public API - if a dependent module is using undocumented stuff then they should pin the version down.
No one is going to see that unless they go looking for it though
>you shouldn't have used "~", "^"
That is how the npm ecosystem works, and in any case the issue arose from dependencies that in turn depended on underscore and thus are not under my direct control.
> Anyway, with a decent test suite, surely a modicum of (automated) testing has already identified the pinning that should take place?
My particular issue is in a new project I could not install several dependencies as npm downloaded the newest compatible version which was supposedly 1.7.0 - the only way to fix this is to manually create a lock file.
>you depend for free.
True, and I'm tremendously grateful for all the work that has gone into it and it would be a massive shame if that work was for nothing if a continued disregard for semver forces people to switch to lodash.
Do you imagine that the maintainers of all these dependencies are going to catch any fraction of the hell that jashkenas seems to be catching for this? It's just as much their fault as it is his. Why don't they pin and test?
https://news.ycombinator.com/item?id=8244888
Do you mean to imply that breaking-change frequency should stay below some particular maximum value? That argument is completely unrelated to the use or disuse of semver.
This issue highlights an interesting philosophical split.
I suspect the same minimalist, intentional aesthetic that has made Ashkenas's projects so beloved equally drives his dislike of version-inflation, whatever its convenience is to downstream systems on autopilot.
Incidently, this is the title the mods chose. Can anyone guess how many spots the link dropped on the front page after that change?
I figured that it was worth a bit of a longer explanation (cross-posted from https://gist.github.com/jashkenas/cbd2b088e20279ae2c8e)
---
Spurred by this thread, here's is a quick set of jotted-down thoughts about the state of "Semantic" Versioning, and why we should be fighting the good fight against it.
For a long time in the history of software, version numbers indicated the relative progress and change in a given piece of software. A major release (1.x.x) was major, a minor release (x.1.x) was minor, and a patch release was just a small patch. You could evaluate a given piece of software by name + version, and get a feeling for how far away version 2.0.1 was from version 2.8.0.
But Semantic Versioning (henceforth, SemVer), as specified at http://semver.org/, changes this to prioritize a mechanistic understanding of a codebase over a human one. Any "breaking" change to the software must be accompanied with a new major version number. It's alright for robots, but bad for us.
SemVer tries to compress a huge amount of information — the nature of the change, the percentage of users that will be affected by the change, the severity of the change (Is it easy to fix my code? Or do I have to rewrite everything?) — into a single number. And unsurprisingly, it's impossible for that single number to contain enough meaningful information.
If your package has a minor change in behavior that will "break" for 1% of your users, is that a breaking change? Does that change if the number of affected users is 10%? or 20? How about if instead, it's only a small number of users that will have to change their code, but the change for them will be difficult? — a common event with deprecated unpopular features. Semantic versioning treats all of these scenarios in the same way, even though in a perfect world the consumers of your codebase should be reacting to them in quite different ways.
Ultimately, breaking changes are no fun, and we should strive to avoid them when possible. To the extent that SemVer encourages us to avoid changing our public API, it's all for the better. But to the extent that SemVer encourages us to pretend like minor changes in behavior aren't happening all the time; and that it's safe to blindly update packages — it needs to be re-evaluated.
Some pieces of software are like icebergs: a small surface area that's visible, and a mountain of private code hidden beneath. For those types of packages, something like SemVer can be helpful. But much of the code on the web, and in repositories like npm, isn't code like that at all — there's a lot of surface area, and minor changes happen frequently.
Ultimately, SemVer is a false promise that appeals to many developers — the promise of pain-free, don't-have-to-think-about-it, updates to dependencies. But it simply isn't true. Node doesn't follow SemVer, Rails doesn't do it, Python doesn't do it, Ruby doesn't do it, jQuery doesn't (really) do it, even npm doesn't follow SemVer. There's a distinction that can be drawn here between large packages and tiny ones — but that only goes to show how inappropriate it is for a single number to "define" the compatibility of any large body of code. If you've ever had trouble reconciling your npm dependencies, then you know that it's a false promise. If you've ever depended on a package that attempted to do SemVer, you've missed out on getting updates that probably would have been lovely to get, because of a minor change in behavior that almost certainly wouldn't have affected you.
If at this point you're hopping on one foot and saying — wait a minute, Node is 0.x.x — SemVer allows pre-1.0 packages to change anything at any time! You're right! And you're also missing the forest for the trees! Keeping a system that's in heavy production use at pre-1.0 levels for many years is effectively the same thing as not using SemVer in the first place.
The responsible way to upgrade isn't to blindly pull in dependencies and assume that all is well just because a version number says so — the responsible way is to set aside five or ten minutes, every once in a while, to go through and update your dependencies, and make any minor changes that need to be made at that time. If an important security fix happens in a version that also contains a breaking change for your app — you still need to adjust your app to get the fix, right?
SemVer is woefully inadequate as a scheme that determines compatibility between two pieces of code — even a textual changelog is better. Perhaps a better automated compatibility scheme is possible. One based on matching type signatures against a public API, or comparing the runs of a project's public test suite — imagine a package manager that ran the test suite of the version you're currently using against the code of the version you'd like to upgrade to, and told you exactly what wasn't going to work. But SemVer isn't that. SemVer is pretty close to the most reductive compatibility check you would be able to dream up if you tried.
If you pretend like SemVer is going to save you from ever having to deal with a breaking change — you're going to be disappointed. It's better to keep version numbers that reflect the real state and progress of a project, use descriptive changelogs to mark and annotate changes in behavior as they occur, avoid creating breaking changes in the first place whenever possible, and responsibly update your dependencies instead of blindly doing so.
Basically, Romantic Versioning, not Semantic Versioning.
All that said, okay, okay, fine — Underscore 1.7.0 can be Underscore 2.0.0. Uncle.
(typed in haste, excuse any grammar-os, will correct later)
> But Semantic Versioning (henceforth, SemVer), as specified at http://semver.org/, changes this to make a mechanistic understanding of a codebase more important than a human one. Any "breaking" change to the software must be accompanied with a new major version number.
I'm about 80% sure you're mistaken on this. I don't have a spec document or an example to back me up, but I've been slinging code for almost 30 years now, and cut my professional teeth on a big Unisys mainframe. Before version numbering in software became totally subjective, they were used to indicate break/change/patch, as described in semver.org.
It was later on that software projects started to move away from that, mostly shareware and desktop software in the 90s, and only recently that projects like Firefox entirely lost the plot.
The reason that "semver" used to be so popular, as you note, is because it made enterprise IT manageable. Developers don't think about that much anymore though; they just expect big IT departments to roll out nightly updates and then deal with the fallout the next day.
(I'm not piling on or picking a fight, I don't use Underscore, I don't have a horse in this race. I'd welcome cited corrections or opinions from even older developers.)
Application software has generally used version numbers in different ways from library software. It's tricky for products like Firefox that are both application and library.
I was under the impression that Mozilla was incrementing Firefox's major version number with every new release, even if new releases didn't have api changes. As described in http://jonoscript.wordpress.com/2011/07/18/its-not-about-the... for example.
Is that not the case? Is there actually some part of Firefox that other software relies on that's getting changed in not-backwards-compatible ways with every release?
But there is a special class of extensions ("binary add-ons") that link to Firefox/libxul as a C++ library. This has been discouraged for a long time now, but still supported for legacy reasons. These extensions have full access to Firefox's internal C++ and XPCOM interfaces. Since Firefox 4, the ABI of this internal library changes with every major update, so every release is breaking for binary add-ons (except security patches, which take care not to change the ABI, and also do not increment the major version). This is mentioned briefly in the comments of the blog post you linked to.
Since that blog post was written, Firefox's add-on system has changed to address this dichotomy. Non-binary add-ons new use "lax" version checking by default (a minimum Firefox version is checked, but no maximum), while binary add-ons still use "strict" versioning. This is documented here:
https://developer.mozilla.org/en-US/docs/Mozilla/Developer_g...
https://developer.mozilla.org/en-US/Add-ons/Install_Manifest...
[disclosure: I'm an developer at Mozilla.]
I wouldn't offer my opinion on version numbering anyway, since my opinion isn't worth anything. It's definitely not in my own interest to argue for sane version numbering schemes; clients pay me hourly rates to fix issues with the software they use.
That may make sense for very large software projects or operating systems where backwards compatibility is a huge concern, but it's not a good fit for the majority of software projects. This is especially true when these projects aren't compiled and thus don't even have an ABI to worry about.
This view on versioning I think comes from a time when most software was compiled and when compiling that software would take a _long_ time or require downtime; thus protecting ABI compatibility was worth the effort.
For modern projects that are in an interpreted language, I think constraining the middle version number to "non-breaking" changes is silly as it's copying a concern that no longer applies. It forces you to choose between "bug-fix" and "feature" releases and pace your project development in a rigid fashion that may not make much sense.
Also, I think it's ridiculous to require anything other than big-fix releases to be non-breaking in the first place. If 1.6.x worked for you, why are you even trying to use 1.7.x if not to access new features? If that's the case, you're going to be editing your code again anyways.
Worrying about a release manager automatically pulling a 1.7 codebase to upgrade a 1.6 codebase is silly; why is the package manager doing this in the first place?
Just one opinion, of course.
A couple comments:
> Node doesn't follow SemVer, Rails doesn't do it, Python doesn't do it, Ruby doesn't do it, jQuery doesn't (really) do it
Node does (because it's <1.0.0 where semver's rules are different). Everything else you list is older than semver, so short of a time machine or a radical change to their long-established versioning policies, there's not much they can do.
> the responsible way is to set aside five or ten minutes, every once in a while, to go through and update your dependencies, and make any minor changes that need to be made at that time.
That works fine for your immediate dependencies. The reason why I think things like semver are important is that the above doesn't work at all with transitive dependencies.
If my app uses foo which uses bar which uses baz and the maintainers of baz decide to make a breaking change in a minor version, I'm not in a happy place when that wolf in sheep's clothing appears in my dependency graph. I'll see that my app is broken, but unless I want to fork bar and maybe foo and start mucking around in code I've never touched before, my options are pretty limited.
I don't think semver is perfect, but I find it does make transitive and shared dependencies a lot more reliable to work with. I'm willing to sacrifice human-meaningful version numbers to get that.
You might've missed the paragraph preëmptively responding to that (it was added in an edit).
SemVer says that you shouldn't be pre-1.0.0 if you're actively used in production anywhere. Node has been out for years. Treating the current versions of Node as pre-1.0.0 is a joke.
From the semver.org FAQ:
> How do I know when to release 1.0.0?
> If your software is being used in production, it should probably already be 1.0.0. If you have a stable API on which users have come to depend, you should be 1.0.0. If you're worrying a lot about backwards compatibility, you should probably already be 1.0.0.
Node definitely applies.
Aside from that, the other 2 indicators definitely do not apply to Node (API is not stable, there is not a lot of worry about backwards compatibility as you'll see in the upcoming 0.12 release).
So no, Node is not '1.0.0' according to SemVer.
Immediate dependencies are a relative thing, after all — foo's deep dependency is bar's immediate dependency. And if SemVer'ing them only works some of the time, but won't save you all of the time, then why bother pretending in the first place?
The actual appearance of breaking changes in software that end up affecting you occurs at the same interval regardless of the version number we decide to label them with. We can do better for determining compatibility than a three or four character version number.
>If the asshole responsible...
If anyone was the asshole it was me, however I truly believe that underscore is better off and I would like to thank you for all the work you have put into it. :)
* We want to be able to easily pick up non-breaking bug/security fixes that should not cause us to change code.
* We want to be able to pick up newer features, and are ok to make changes to our own in response.
In both cases, we need to be confident about which sort of upgrade we are getting.
Semver gives that; it's just unfortunate that it doesn't map to the human notion of what a major version is.
And you don't fight battles without a deal of incoveniencing. The mere fact of the publicity this is generating makes it a fair ground to fight that battle. Which is to say I can see his position, despite liking semver myself--though it could be distilled into a better standard and that could be a net positive long term on software development that far offsets the man-hours now being lost.
The only reason for having automatic package updates in the first place is to receive bug and security patches. There is little gain in receiving 'minor' updates if you are not changing your code, and any serious project will end up freezing the dependency tree anyway.
https://github.com/npm/npm/releases/tag/v1.4.3
[look at the patch number...]
If you have a lot of breaking changes, you probably shouldn't be at 1.x.x. If you need to deprecate something, add the extra features alongside and remove the deprecated features in the next major release. If you need to add something to code that is >= 1.x.x and you think it will need breaking changes, you should mark the feature as experimental in the documentation until it has stabilized. Semver works great for humans and robots if you put a smidgeon of planning and consideration into the process!
E.g., 1.1.7.0 or 1.7.0 are equivalent.
[Cross posted from the other thread: https://news.ycombinator.com/item?id=8244920]
It would make a big difference if they were ordinary users, vs maintainers of other important libraries of pieces of software.
While everyone deserves their say, this issue seems somewhat subjective, and a package maintainer shouldn't have to pay undue attention to individual users who don't necessarily represent the whole community.