Hacker News Reader

Comment by ChrisMarshallNY | Hacker News Reader

> I’ve personally been burned on Python by the last bullet point where one of the dependencies required version 3.14 or less of module such and such, and another dependency needed 3.15 or higher.

This is what people used to call “DLL Hell,” in Microsoft Windows.

COM was supposed to fix that, but I don’t think it worked especially well.

View on HN

COM has nothing to do with fixing DLL hell, it is as old as OLE, and uses DLLs as well.

COM fixes having a cross language OOP ABI, that is all.

The things that were supposed to fix that were:

1- Version resources, which hardly matter, because the app loading them has to validate the versions itself

2- .NET, hence the Global Assembly Cache and having the version as part of the Assembly (DLL) lookup. Still has issues if the lookup rules were badly configured on the app.

3- Application manifests, bringing the .NET ideas to Win32

4- Registry Free COM, extension of app manifests, allowing direct lookup of desired COM libraries without going through the registry for the version

5- UWP sandbox, didn't took off, only what is inside the sandbox is searched for

6- Easiest one, don't put stuff all over the place, only search inside an application specific directory

Rust fixes the problem at the language level by allowing your dependencies to use different versions of their dependencies. It can be a little rough on the size of your binaries, but you can end up with as many different versions of the common dependencies as you need.

The issue is probably the same as with COM. COM introduced versioned “packages,” inside of the library, but that meant that library providers needed to package up multiple versions, as part of their CD system.

In practice, a lot of orgs just stopped including deprecated versions.

Yea, that was a mistake. Rust handles it transparently; if the program you write happens to depend on two versions of something, it just downloads them both for you.

How does this work with linking?

I would assume that both versions publish the same API, so does Rust add a mangling to the link reference?

One of the things that made DLL Hell so bad, was that you would have no idea that you were calling a deprecated/new version of the function, until it started misbehaving.

There is no dynamic linking in rust when it comes to other rust packages.

Rust does name mangling, and it basically just adds the crate version to the mix when it mangles the names. So if crates A and B both depend on X but different version, then A can only call functions from its chosen version of X while B can only call functions from its version. There is an optimization pass that discards duplicate functions, so functions in X that haven’t actually changed between versions will be deduplicated. This all happens automatically, so nobody ever has to think about it. Not the crate authors, and not you.

What does get complicated is when the crates A and B both deal in types from X. If a function from A returns a type from X and a function from B takes that same type as an argument, then the compiler will step in with an error that the types don’t match. It’ll tell you that type Foo (from crate X version 1.0) doesn’t match Foo (from crate X version 2.0). This prevents all the possible runtime errors that could occur if you were really mixing an matching between both version. In that situation you will likely need to constrain your chosen versions of A and/or B such that they can agree on a single version of X, instead of allowing cargo to simply pick the latest available version.

I'm sure that cargo does a clever job of all of this but this kind of functionality is precisely why I find rust so off putting. It encourages you to take on huge amounts of unnecessary complexity (including complicated dependency trees) and then tries to hide that complexity in abstraction. But in practice these are always leaky abstractions that _someone_ (and likely you) will have to pay for. At a baseline, the poor compilation times and byzantine of rust are to me the most obvious symptoms of this embrace of complexity.

If the language you use don’t have any way of dealing with this problem then one day you will be dealing with heap corruption because a library you (transitively) depend on changed the size of a struct. This is the fate of every C and C++ programmer.

Or you could use a safe dynamic language like Lisp, Javascript, Python, and many others. Programmers using these languages are so productive because they never have to put up with any nonsense like random heap corruption.

Or you could use Rust, which gives you the safety without any of the runtime costs incurred by the dynamic languages. What you call Byzantine complexity is in fact a very simple rule that you can teach to anybody, and which in practice most developers never need to deal with. It doesn’t even slow down the compiler.

Engineering is all about tradeoffs, so choose wisely.

That only works if the dependency doesn't expose types used by multiple other dependencies.

I keep running into this with the Rust 3D graphics stack. That's winit (windowing), wgpu (wrapper for Vulkan, Metal, WebGPU, etc.), egui (2D menu overlay), and support crates such as wgpu-egui, wgpu-profiling, tracy-profiler, and glam (vector and matrix math). Every time one of these has a breaking change, it takes 4 to 6 weeks before the whole stack works without patching. Then I have to fix three more levels of my own.

Yep. In principle you can examine the crates before you use them and pick ones that won’t ever expose you to that pitfall. In practice, of course, things are not always so ideal.

But at least if you end up with incompatible versions then the compiler will step in and prevent your program from compiling, instead of allowing it to crash mysteriously later on. It’s frustrating, but not half so frustrating as it could be.

Yes. Cargo and the Rust compiler do catch clashes.

A big advantage of that is that you seldom have to do a clean build in Rust. Make-based systems seem to require regular "make clean" cleanups, but Cargo has enough smarts to really know when it can avoid recompiling.

Not quite. DLL Hell was the situation when apps installed their dynamically linked dependencies globally, so that when two apps shipped different versions of the same library they could conflict. The situation was much rarer for dependencies because dependency trees simply weren't that deep in that era.

It had also been solved since the introduction of assembly versioning over 20 years ago, which provided a standard way to have multiple versions of the same dependency installed and loaded side by side. Later Windows versions also added the ability to do the same for app-local DLLs, to the point where you can have different parts of your code get different versions of the DLL when they ask for it within the same process. So, for all practical purposes, it has been a solved problem for a very long time now.

It required the re-implementation of stable interfaces as the components updated in version number; not everyone did that. I guess. A case of good plan, lazy execution.

DLL hell hasn't been a thing for more than a decade at this point though, so maybe it did. I genuinely never encounter any issues with DLLs anymore.

An awful lot of it went away in my experience in the c++11 world when everyone all Of a sudden wanted to upgrade. Then another time in 2015 when MSVC decided their standard library would be backwards compatible

Most Windows apps ship with copies of all their dependencies. The DLL Hell problem was solved by disk and bandwidth getting cheaper to the point where many dozens or even hundreds of copies of MSCVRT.DLL and friends on your system pass unnoticed.

But isn't that a good thing? I don't mind having more DLLs if stuff just works. It avoids the similar, though harder to solve issues on Linux. I'd rather be able to have a copy of the MSCVRT.DLL than to have to pray that my system has the right glibc to be honest.

But maybe I'm not understanding the trade offs correctly.

> I don't mind having more DLLs if stuff just works.

The counterpart is when a remotely exploitable security issue is found in that DLL, as has happened with zlib in the past and log4j more recently; then you have to chase down and update every single copy of that DLL.

If you absolutely need a way to install dependencies globally such that they can be shared between apps, you can still do that without version conflicts if you write proper manifests for them and use https://en.wikipedia.org/wiki/Side-by-side_assembly.

But it's easier to just package a local copy and not worry about any of this, so that's what people mostly do now.

The cost of storage space these days is about $0.01 per 100 megabytes.

Then later on NuGet was supposed to fix that, but even with a package manager you encounter "dll hell" like scenarios.

What I found mostly curious is I didn't even knew phyton 3.15 had already released

The example was talking about versions of another package that both of the given packages depend on. Not Python itself. The version numbers were made up.

Reply on news.ycombinator.com