Hacker News Reader

Comment by brabel | Hacker News Reader

The author added a lockfile without understanding why they exist.

A lockfile is meant to "freeze" dependency version resolution when package authors can specify dependencies on other packages using version ranges... it also "freezes" choices of transitive packages' versions when different packages depend on the same one, but with different versions.

They chose to not handle package dependencies at all, and I believe there's no version ranges either, so I really don't see why they added a lockfile.

View on HN

There is quite a difference between "the author added a lockfile without understanding why they exist" and "I really don't see why they added a lockfile".

It's better to start with the latter if you seek to understand, not attack.

FWIW, cynically speaking, more often than not "I really don't see why ..." roughly translates to "You don't know what you are doing ..." in business dialogue

The lockfile in this post actually seems a bit more like a manifest, or at least that it's trying to do both things at once.

I'd expect you'd have human readable file to list your dependencies and versions in a simplistic way, and then a separate machine readable file that locks the versions for reproducibility. As it stands, the example in the article does not look pleasant to write by hand so you'd have to script something yourself.

A version number is just a label, and labels are mutable. A lock-file containing hashes will always resolve to the same packages (or fail).

Depending on a single specific version is wrong. A library can be changed to fix a bug without changing the interface in any breaking way. It should be possible for a user (also their package manager, automatically) to replace the library with the new version in this case.

Depending on how important supply chain security is to your industry/company/team, validating the hash of every package is critical. If an attacker can manage an interception/man in the middle attack on your CI network, the hash check provides protection. If an attacker compromises the server you get packages from, having a hash provides protection as well. Automatically trusting the server's response to be correct, or automatically upgrading to newer versions (even if the package author strictly adheres to SemVer), puts you at risk at attacks similar to what we saw with SolarWinds around 2021.

Your argument supports the idea of getting rid of the lock file and instead committing the hash in the original dependencies file - so that it's never an automatic process to update the hashes/versions.

Zigs package manager takes this approach.

https://zig.news/edyu/zig-package-manager-wtf-is-zon-558e

No, it isn't.

Just because someone says X does Y, it doesn't mean it does.

For anything serious you absolutely verify checksums. Ideally you also mirror every dependency used so you don't care anymore about what's out there.

The thinking in your comment lead to Maven range and npm general atrocities.

Every time I download someone's code I replace all the == requirements with >=s and it works perfectly (I understand there are many cases when it wouldn't).

Every time an old unmaintained Linux app I need fails to start, saying it needs some libsomething.2.3 which isn't in the repos already I just symlink the libsomething.2.5 to it and it works great.

Some times this even helped me to overcome bugs/vulnerabilities.

Being able to fix a bug and update a library without the program even knowing (whithout having to get and rebuild the source or contacting the author) is why dynamic linking has been introduced in the first place, isn't it? Isn't this the "unix way"? Is having a program superglued to an outdated library with known (and fixed already) bugs really what you want?

The Unix way is not gospel, and even if it were, this is Computer Science, not Computer Faith :-)

It depends on what you want to do.

If you're hobby hacking stuff, sure.

Any kind of software that's supposed to come out of software engineering, probably not.

You are mad.

I love you.

> Is having a program superglued to an outdated library with known (and fixed already) bugs really what you want?

It is not superglued. If you want to update dependencies, just remove the lockfile and reinstall everything. The main reason people do this is because just updating a library without the program knowing by not specifying the exact version leads to behaviour silently changing, which is terrible (especially on CI!)

It's not wrong to depend on a single version, just suboptimal.

Exhaustively testing every combination of dependencies with your project is infeasible for any non-trivial set of dependencies. The "is compatible with" relationship present in some package managers (e.g. ~= in pip) isn't guaranteed to work because packages can "lie" about their compatibility.

Sure, it's better to have some kind of way for a piece of software to specify that it depends on foo 3.x instead of tying it to a specific release, but at least being able to specify some dependency is better than not at all. In the worst case, the user can treat the "required" version as a suggestion/guide, override that version with whatever they want, and see if the code works.

Why not just replace the "requires" semantics with "has passed tests with" and "has failed tests with" and let the host decide?

In general: no reason, I quite like your idea and think that it's useful. However, in the specific case of the blog post: because that's more complexity, which requires more effort.

I suspect that the author would have added in a more complex versioning/dependency system if he had more time/energy. But, given that he didn't, he had to make a choice about what features to include, and how much to flesh out the various systems.

A simple "depend on exact version" system is relatively easy to implement, and still (indirectly) supports users overriding the lockfile by manually editing it. A more complex (but flexible) system like pip's or yours would be more ideal from a user's perspective - the developer just needs to put in the time to implement.

However, I don't think that any of the above systems are wrong, unlike e.g. one that only allows you to specify a specific version in the lockfile, but then silently downloads "updates" to those dependencies in the background.

This is basically what I daydream about for Nix some day (Nixpkgs doesn't currently involve any dependency resolution at all). Just recording successful version combinations (both of a package declaration and source version and dependency versions) would let us add versioning metadata and save historical package versions in-repo (or in an ancillary repo) without actually changing how the monorepo is built/used.

If you’re going for guaranteed repeatable builds (for example you’re using Bazel) then you have no choice. Dependencies must have a single resolution.

Reply on news.ycombinator.com