Comment by simonw | Hacker News Reader

Comment by simonw | Hacker News Reader

Parent Full thread

Breaking changes in requests would be a disaster.

The problem is that they're not just a library people use directly: they're a library that is often used by other libraries.

The worst possible form of dependency hell is when you want to use libraryA - which depends on requests<3.0 - and libraryB, which depends on requests>=3.0, at the same time.

At that point, you're simply stuck. You cannot use both libraryA and libraryB at the same time until they both upgrade to the same underlying requests dependency.

Python projects are affected by this because it isn't currently possible to have two versions of the same library installed at the same time, because they clash with each other in the sys.modules global namespace.

This isn't true of all modern programming languages - I've been meaning to dig around and figure out which ones are affected by this problem and which are not - I think Node.js and Rust have mechanisms for this, but I don't think Go or Java do.

> I don't think Go or Java do.

Go's module system was specifically designed with this problem in mind: https://go.dev/blog/versioning-proposal

I think Java projects get around this when they have to with shading, but that's a bit clunky.

Java had some magic classloader ideas back in the day. OSGi, I think it was? I don't remember them using the term shading, but I also don't remember ever liking how that went. Usually very confusing for all involved. (Unless you were on the happy path, I think.)

It can be handled "kludgingly" by incrementing the name. e.g. foo --> foo3

Not sure why that's "kludgey".

Dart doesn't have this problem.

There is a global cache of downloaded packages, but it's just a pure cache and happily stores multiple different versions of the same package.

Each Dart application (or package) has its own set of selected package versions independent of any other application on the same machine.

Can you run two versions of the same Dart library/package within the same application?

Python (thanks to virtual environments) works fine at multiple package versions, it's only when they need to be running in the same application process at the same time that you run into problems.

No, by design within a single program you only ever have a single version of a given.

Because Dart is used primarily for client-side applications (and initially only web applications), we care a lot about code size. NPM's approach of silently allowing multiple versions of the same package adds a lot of bloat, which is fine for a server language but not great for a client one.

Also, nominal static typing makes having multiple versions of the same package potentially very unpleasant for users. Say you have an application that depends on packages "a" and "b". "a" and "b" both depend on a package "foo" and each gets their own version of "foo". The "foo" package (both versions) define a type Foo. It's entirely possible for an instance of Foo from "a"'s version of "foo" can flow through the program over to "b" and then get passed to "b"'s version of "foo".

The end result will be compile time and/or runtime errors like "Expected a value of type Foo but got a value of type Foo." This is not a great user experience. NPM avoids this problem by being dynamically typed. Go mitigates it by being structurally typed for interfaces.

Obviously, better error messaging can help, but it's just generally confusing to have multiple versions of what a user thinks of as "one" package when really there are multiple floating around in the same program.

For the most part, selecting a single version of any shared dependency works pretty well. It's definitely not perfect. It can make it harder for heavily used packages to make breaking changes. But the overall trade-offs seem to work fairly well.

npm you either have as many copies of a dependency as its used, or you flatten it and have the same problem.

go codifies foo -> foo2 as foo -> foo/v2

NPM and other Node.js package managers happily deduplicate dependencies are compatible within the specified version constraints. So if 3 different modules in your build specifies `foo^2` and two other modules specify `foo^3`, in general you will have two copies of `foo` loaded.

Reply on news.ycombinator.com