On breaking changes in transitive dependencies | Hacker News Reader

On breaking changes in transitive dependencies

dmytro.sh

24 pointsluu22 comments

> That change is absolutely trivial to handle, and it will be a while before the behavior change. It doesn’t require a major version bump.

That change, however trivial to handle, is still a bloody breaking change and it still has to be handled, by the library users, at one point of time or another. And how do you notify your users that they need to handle something that's changed? That's right, you mention it in the RELEASE file, a major version bump is entirely superfluous. /s

This is why we can't have nice things: apparently there are library maintainers out there who just don't get it what semver is actually about.

This:

"Imagine a world where software could be tested against a version of the dependency, and then allow all version updates as long as they are minor (a new functionality added in a backward-compatible manner)"

is a fantasy in my experience. The Semver contract exists as a concept sure, but there's no guarantee that the publisher of a dependency you consume actually maintains compatibility, in the context of your applications usage, between minor versions. It's more of an aspiration.

As someone who has been bitten time and time again by transitive dependencies not adhering to semver while still claiming to be using semver, i totally agree. Dependency pinning is a must when you want to have full control, and even then some package managers allow pulling or updating specific versions of dependencies (looking at you, pip).

One can conceive of a solution that bisects dependencies independently to find the latest version that satisfies a comprehensive test suite. But that build and test is going to have to be reasonably quick. Even moreso if you take the exhaustive approach and transitively apply the algorithm to dependency dependencies which would also require backtracking.

A more durable approach might be to require every API to formally and verifiably provide precondition and postcondition that can be checked with an SMT solver for every public interface.

With current tech that is pushing a lot onto module authors, but I believe there is some research being done on inferring both preconditions and postconditions from the semantics of the component statements so perhaps we can avoid forcing programmers to carry the intolerable burden of learning the first order predicate calculus.

I haven’t figured out the math yet, but on my shelf if future projects, I have a part of a plan to tie semver to unit tests.

The theory so far is: there’s a subset of unit tests that specify the API. If you create new tests, that’s mostly fine. If have to alter a test, that’s probably a breaking change.

The main problem I can already see with this, which is why it’s on the shelf, is that some test frameworks and many tester writers cannot guarantee the invariants as stated above. You end up changing tests for minor behavioral changes. You mix tests of private fields in with tests of public ones. And even if you split them, then functional tests end up defining parts of the API.

But somewhere out there in the future is a language where types are defined by tests, and certain testing patterns indicate minor version increments and others major.

I've thought about this too.

The other issue is that tests can have bugs too, sometimes you are just fixing a test bug.

Definitely. So in a robust system, you might also have to look at code coverage to see if the test interacts with any code changes since the last minor version increment.

If I change the test but not the code, that may be a clarification. If I change both, it’s ambiguous.

This has bitten me twice before: a transitive dep update broke my library. I now publish my npm shrinkwrap file and only officially support npm (or anything that respects the published lock file, which I think is only npm).

I think yarn and pnpm can read these

They don't use published lock files.

Why do you prefer shrinkwrap instead of npm's package-lock.json?

Package locks are ignored by npm when published.

Oh, so do I understand it right if you want your users to use specific pinned version for all of your dependencies? In that case why not pin to specific versions (as opposed to semver ranges) in package.json, which would be respected identically by npm/yarn/pnpm and again remove the need for shrinkwrap?

Yes. Because that doesn't pin dependencies of your dependencies (this is what "transitive dependencies" means).

Would you like to do the same thing for Truffle/Ganache or are there non-political reasons why you don't find this approach suitable for either or neither?

I do this for the main `ganache` package, which is also mostly webpacked (only because of the install-time improvements it enables, and for easier browser usage), but not the other monorepo packages (they are all still pre-1.0.0 versions anyway, so I'm not as concerned with API or transitive dep stability).

Truffle, which I don't work on myself very often, is more trusting of semver, and does use yarn for development.

I in the past 've had to step away from a Thanksgiving dinner to do a hot fix release due to a transitive semver breakage, so I'm probably more sensitive to its problems than others.

Yarn ignoring published lock files isn't totally crazy though. Yarn states that allowing automatic updates to transitive dependencies is better because users can respond to new security incidents faster than a chain of downstream package authors. An excellent point. My stance is that package authors generally introduce new breaking bugs/exploits much more often than they fix them; popular tools like Ganache are more likely to be the target of supply-chain attacks than non-financial tools.

I'd honestly love to switch to, or support, a version of yarn, or pnpm, that allows package authors to prefer strict transitive dependencies, that can then be overruled by consumers of the package of they prefer a looser policy.

My hot take is that SemVer can only really work if the team also restricts how much of the existing code they are willing to change. In particular, the point is stability of existing code, which means that you basically have to somewhat restrict yourself to use the existing ways as much as you can. Any codebase where folks are building new ways of doing things will build up odd mismatches in what is actually supported versus what exists, and then gives incentives for folks to remove old ways.

Worse, SemVer is useless from small teams where they are not budgeting time and effort to maintain all of the old version bumps. Pointedly, just updating the feature version can help folks know it is dangerous to upgrade. But, if you aren't going to maintain the old versions forever, it is equally dangerous to not upgrade.

No, Semantic Versioning (SemVer) is not a silver bullet. But the concept can be an effective software stabilizer. However, utilization of unbounded versions in transitive dependencies (for example, `pandas>=1.5` in a python requirements.txt) can silently break this contract. Even if a transitive dependency utilizes SemVer, a consumer of such a library has to also conform. The pandas example was found in a transitive dependency in one of our production processes, and one day this April it broke our production service when pandas released its 2.0 version. We had pinned all our dependencies, but the man in the middle had not. The python ecosystem is notorious for this time consuming naiveté.

One possible issue is that compile or test time dependency resolution, especially for uncompiled languages, doesn’t distinguish betweeen explicit and transitive dependencies in most package managers/build systems. If you have a direct source dependency, imho, it should be explicit (not relied on transitively), and if not (unless a specific runtime version is necessary), transitive dependencies should not be locked or specifically annotated in a lock file. The former is overly flexible, the latter overly rigid.

I also prefer an external manifest that can lock specific versions of all dependencies, but that usually requires a separate package manager from the language provided one, which introduces a whole new set of issues.

This seems like a case where Go’s ‘minimum version selection’ approach would have helped——Sidekiq would have continued using an older version of the dependency that didn’t have the breaking change.

Integers used for errors, lack of static typing, no semver, no tests to verify that behavior. I guess Ruby also can't use multiple versions of the same gem(?).

It is good to hear that the Ruby ecosystem is improving advocacy and utilization of Semantic Versioning given, in my experience, its unstable past.

Reply on news.ycombinator.com