Hacker News Reader

Comment by lucian1900 | Hacker News Reader

lucian1900

Multiple versions may sound like it's useful, but it's almost always a bad idea. Cargo doesn't allow it either.

The problem isn't really fundamental. Bundler makes almost all the right choices already. Its major disadvantage is that it only works for Ruby.

View on HN

As a practical matter, the npm ecosystem today relies on duplication, and no new client that made the "highlander rule" (there can be only one) mandatory could succeed.

Yarn does offer a `--flat` option that enforces the highlander rule, and I'm hopeful that the existence of this option will nudge the ecosystem towards an appreciation for fewer semver-major bumps and more ecosystem-wide effort to enable whole apps to work with `--flat` mode.

Plz send halp!

Explain why duplication is mandatory?

Just imagine two packages you depend on (a and b) that both have a shared dependency (x). Both start off depending on x version 1.0 but then later a is updated to 2.0 while b isn't. Now you have two packages depending on different versions of the same package and hence the need for duplication. You have a that needs x@2.0 and b that needs x@1.0, so both copies are kept.

Don't upgrade a when it wants a half-baked x. Choose versions of a and b that agree on a known-good version of x. If there aren't any, it's not sane to use a and b together unless x is written very carefully to accommodate data from past and future versions of itself.

That's not how it works. There will be two copies of x in the require cache. They don't know of each other's existence.

I'm arguing for choosing dependency versions that don't require you to break the highlander rule. "a is updated to 2.0" doesn't mean you should start using that version of a right now.

Right but what I'm saying is that two versions of the same library will live quite happily together, because node.js absolutely abhors global scope. The two versions have their own module-level scope to store functions and data.

So go crazy and install every version of lodash! Nothing will break.

It's almost certain that a's version of x and b's version of x have distinct types whose names collide, and I don't know of any Typescript or Flow notation to prevent passing incompatible instances between them (e.g., a wants to call a method that didn't exist in b's version of x), so if anything works it's only by luck.

Edit: I haven't dug into this, but it might be possible to use Typescript namespaces to distinguish a and b's versions of x. https://www.typescriptlang.org/docs/handbook/declaration-fil...

Node's module system doesn't import dependencies into a global namespace. So the `x` required by `a` and the `x` required by `b` will have separate namespaces and won't conflict, or even know of each others' existence. There are pros and cons to this approach, but it definitely does work. Flow, at least, (and presumably Typescript) understands the module resolution logic and handles this very common situation without issue.

It's not possible to import two different versions of a module in the same module.

If a depends on x v1 and exports this dependency then your application also imports x v1. If b depends on x v2 and exports this dependency too that means your application is transitively importing both x v1 and x v2 which is not possible.

If a or b only use instances of x internally and do not accept instances of x as parameter or return an instance of x they don't have to export their dependency on x.

If either a or b do not export their dependency on x then there is no problem. Your application will only depend only on x v1 or x v2 directly.

That will work for lodash, because it's just a bag of functions. But anything that creates objects that are passed outside of a single module could have problems.

There was actually a high profile bug when someone ended up with two versions of react included in the same page: https://github.com/facebook/react/issues/1939.

That seems to be a slightly different situation? As the issue poster described, a single page had invoked two copies of React, both of which assumed ownership of a particular "global" variable. (Not really global, but a particular member of a global.)

I'm not a React expert, but I don't see why that situation would only affect multiple React copies with different versions?

I'm not a large JS developer but with my other experiences managing dependencies, it doesn't always allow this as a possibility. A bug in your system is tracked down to the library b using x@1.0 but the fix is to upgrade to b with x@2.0 however a using x@1.0 doesn't have an upgrade path. Waiting for another company or organization to release with an update is not an option. Our projects have several cases of requiring different versions of the same library -- we try to avoid this using the same logic you suggest but it's not a possibility in all cases so we have to work with it.

It's placing your own release cycle at the whims of your dependencies' release cycles. In the corporate world that would not be a viable solution.

It's not as simple as that. Lodash is a great example of why the highlander rule doesn't work within the npm ecosystem: older versions are depended on by many widely-used packages which are now "complete" or abandoned. Refusing to use any packages which depend on the latest version of Lodash is just not practical.

Would it be possible to create hardlinks or symlinks to a particular package/version pair shared as a dependency between other packages? I know this only works on unix-like OSes but otherwise it could revert to the old behaviour of duplicating the dependency.

I think they're just saying that any new client that tried to not support duplication at all would likely quickly run into a large amount of npm packages/package combinations that just don't work. So within the context of using the npm registry duplication is mandatory.

Cargo definitely allows two dependencies to rely on different versions of a transitive dependency. If those deps expose a type from that dependency in their API, you can very occasionally get weird type errors because the "same" type comes from two different libraries. But otherwise it Just Works(tm).

Cargo does allow multiple versions of transitive dependencies. It tries to unify them as much as possible, but if it can't, it will bring in two different versions.

What it _won't_ let you do is pass a v1.0 type to something that requires a v2.0 type. This will result in a compilation error.

There's been some discussion around increasing Cargo's capacity in this regard (being able to say that a dependency is purely internal vs not), we'll see.

With tiny modular utilities this is very much necessity - and not a bad idea if the different versions are being used in different contexts.

For instance, when using gemified javascript libraries with bundler it is painful to upgrade to a different version of a javascript library for the user facing site while letting the backend admin interface continue using an older one for compatibility with other dependencies.

You've got to take the ecosystem into account. There are a lot of very depended-upon modules that are 1 KB of code and bump their major version as a morning habit. Forcing people to reconcile all 8 ways they indirectly depend on the module would drive them nuts, but including 4 copies of the module doesn't hurt much.

Yarn supports `yarn [install/add] --flat` for resolving dependencies to a single version

> Multiple versions [...] almost always a bad idea

If so, different major versions of the same dep should be considered different libraries, for the sake of flattening. Consider lodash for example.

That's exactly what Cargo does, but it also takes a further step of making `^` the default operator and strongly discouraging version ranges other than "semver compatible" post-1.0.

Looks like Yarn does something similar: https://yarnpkg.com/en/docs/cli/add#toc-yarn-add-exact

Personally I'm not really sure I like it. If I specify an exact revision of something, chances are I really do mean to install that exact revision. I don't see why I need an extra flag for that.

That can cause some serious problems in at least some portion of times. I've dealt with the subtle errors that have been caused by this problem in c++, and don't really know javascript libraries that well so I can't give a more concrete example. But imagine that there are the following libraries:

* LA: handles linear algebra and defines a matrix object.

* A: reads in a csv file and generates a matrix object using LA

* B: takes in a matrix object from LA, and does some operations on it

In this case, if B depends on version 5 of LA and the new version of A depends on version 6 of LA, then there's going to be a problem passing an object that A generated from version 6 and passing it to B which depends on version 5.

The problem does happen in JavaScript. But since its unityped, there is a strategy to deal with it

* Figure out early on (before 1.0) what your base interface will be.

For example, for a promise library, that would be `then` as specified by Promises/A+

* Check if the argument is an instance of the exact same version.

This works well enough if you use `instanceof`, since classes defined in a different copy of the module will have their own class value - a different unique object.

  * If instanceof returns true, use the fast path code (no conversion)
  * Otherwise, perform a conversion (e.g. "thenable" assimilation) that
    only relies on the base interface

Its not easy, but its not always necessary either. Most JS libraries don't need to interoperate with objects from previous versions of themselves.

Would this even work in what I describe? For instance, if mat.normalize() was added in LA-6, and B provides an LA-5 mat, and then A (which has been updated to use the new method) calls mat.normalize() on the LA-5 mat expecting an LA-6 mat but because of duck-typing that method doesn't exist.

It would not.

However, since A exposes outside methods that take a matrix as an argument, it should not assume anything beyond the core interface and should use LA-6's cast() to convert the matrix.

The problem is partially alleviated when using TypeScript. In that case the inferred type for A demands a structure containing the `normalize` method, which TypeScript will report as incompatible with the passed LA-5 matrix (at compile time). That makes it clearer that `cast` would need to be used.

If an API expects the outside world to hand it an instance of a specific library, all bets are off. Maybe it gets `null` or the `window` object, who knows? But a library can at least declare what dependencies it wants. If you take that away, it ratchets up the uncertainty factor that much more.

Define the matrix class in a seperate library for compatibility purposes if it's so widely used and doesn't change. Maybe some people don't need both the linear algebra and instead only the definition of the matrix object?

Another solution is to provide version overrides and make B depend on version 6.

However if there are differences in the matrix class between different versions of the library then you're forced to write a compatibility layer in any case.

This is especially true when you're going to be serving your code over the web. It's very easy when using npm to accidentally end up shipping a lot of duplicate code.

That alone has me super excited about Yarn, before even getting into the performance, security, and 'no invisible dependency' wins.

I feel like this is especially problematic when using NPM for your frontend. Now you have to run all your code through deduplication and slow down your build times or end up with huge assets. I wonder if it's really worth the trouble.

Reply on news.ycombinator.com