Hacker News Reader

Comment by kstrauser | Hacker News Reader

What's all that about the bcachefs author's complaints with Rust and Debian? I'm far out of the loop on this stuff.

His side is that basically it doesn't make sense to package the tools for an experimental FS in an OS that's going to get very far behind, as debian stable will do, since the tools have to iterate rapidly to fix problems with releases of the FS. Debian-stable had an old set of some rust libraries and the packager relaxed the dependency constraints in order to package it, which doesn't make a lot of sense on something you are hoping to fix your filesystem.

Basically it shouldn't be packaged for LTS-class OS releases until it's not experimental.

Ah, I see. So "the Rust team" here means the team of Debian maintainers who package Rust for it, not the Rust developers? That makes sense.

Yeah I assume that's what "the Rust team" meant

This makes perfect sense to me. Why would Debian-stable include something obviously not stable. They have an experimental branch for a reason

To my knowledge, Debian changed deps in bcachefs-tools to synchronize with Debian's Rust repo, breaking it. It's one part that it's a green fs, and the other with clashing expectations of how to treat dependencies between Rust and Debian.

> [bcachefs auhor's] side is that basically it doesn't make sense to package the tools for an experimental FS in an OS that's going to get very far behind

If the bcachefs author's believe the tool is too unstable for Debian stable and Debian developer's believe bcachefs is too unstable for Debian stable [1], it sounds as if they agree.

[1] https://jonathancarter.org/2024/08/29/orphaning-bcachefs-too...

Kent's issue IIRC was that they loosened the dependencies, full-stop. Debian presumably isn't replicating his entire testing regime when they change all the dependencies being compiled in. The potential exists for a bug to be introduced that is genuinely consequential to the user, moreso than for, say, a broken calendar app. Combine that with a rapidly changing FS and the fact that any issues would likely be blamed on his FS and I can see why he might feel that way.

If a distro chose to build some program with a different set of dependencies than what is specified by its author, then arguably it is not the same piece of software anymore. If Debian wants to do that - which I think they 100% have the rights to do, this is free software after all - they should make it clear that they will also take over the maintenance responsibility for it as well.

Distros do this all the time with C libraries. Until newer languages added support for lock files and stuff, it was pretty normal to use slightly different dependencies.

Lock files _are_ code. Same for meson.build or CMakeList.txt. We don't use to have the ability to precisely specify dependencies in code, now we do.

Building with a patched codebase is effectively building a fork.

> they should make it clear that they will also take over the maintenance responsibility for it as well.

Don't they already? I was always told that if you hit a bug in a distro package, you report it to the maintainer, and then if applicable they can pass it upstream. The whole point of a distro (at least the kind Debian is) is to be its own thing.

Oh, if only.

A large part of the issue here was the snafu resulted in Debian users not getting updates for months for a critical fix, and then not being able to mount in degraded mode, and I was the one fielding those bug reports.

Okay, so ask if they're using your packages or the distro packages, and if it's the latter tell them to talk to the person who maintains those packages. If it was me I would have that be part of the bug report form up front, but I don't know what your process is.

That's just passing the buck, it doesn't do anything for the users who were affected by the very real screw up

It also doesn't work in practice because I have to do most of the diagnosis before I find it if it's my bug or Debian's.

The only real solution is for Debian to be better at working with upstream, and not do things they've been told are going to cause problems, and not drop the ball when they do.

> It also doesn't work in practice because I have to do most of the diagnosis before I find it if it's my bug or Debian's.

No, you ask that first and if it's a downstream package you stop working on it. If the downstream maintainer determines that it is on your end and not theirs, then you can pick it back up.

> The only real solution is for Debian to be better at working with upstream, and not do things they've been told are going to cause problems, and not drop the ball when they do.

Assuming that "working with upstream" means "adopting upstream code in direct contravention of their own policies": If your solution depends on Debian not being Debian, then it's unlikely to work.

> No, you ask that first and if it's a downstream package you stop working on it. If the downstream maintainer determines that it is on your end and not theirs, then you can pick it back up.

They're not going to devote that kind of time. That just means bugs not getting looked at or fixed.

> Assuming that "working with upstream" means "adopting upstream code in direct contravention of their own policies": If your solution depends on Debian not being Debian, then it's unlikely to work.

I'm not sure why you think that policies that are causing problems shouldn't change.

Again: they volunteered to package it, they did it badly and users were affected. Until they can get their act together, bcachefs-tools won't be in Debian.

That's ok. There are other distros, and there's no rush.

Distros generally do try to do this, but it doesn't matter much in practice, not enough users go to the distro by default (in part because the distro maintainers, quite understandably, can't actually solve most of the problems, even the ones they caused). So a distro that releases a broken package inevitably causes a support burden for upstream.

(I personally agree that debian's "let's just relax version dependencies on rust projects and hope that it works" policy is insane: a recipe for all kinds of subtle breakage that no-one working upstream is even trying to avoid. And this kind of fiddling is not unique to rust, either, nor is bcachefs the first project to say "don't use debian's packages")

Debian would compile dependencies with versions lower than specified in the project, reintroducing bugs that users would blame upstream for, https://www.phoronix.com/news/Debian-Orphans-Bcachefs-Tools

Got it. I could see that being an issue.

Why is Kent not providing his own deb packages that users can install to override the Debian provided ones to get updates?

Because he isn't a Debian packager. That isn't his job.

Perhaps down the road, after experimental is lifted. For now I've generally been telling distros to slow down.

For anything this big staging the release is important, we don't need or want to be in every distro right away (the Fedora people were also _extremely_ gung ho in the past and I told them the same thing).

Until every outstanding critical bug report is fixed (and there are still a few), I want power users testing it who know how to deal with things breaking, I don't want unsuspecting users getting bit.

Already using it on a 3 drive gaming PC on NixOS and it's been great so far. The caching algo massively speeds up interactions and load times.

Hope you continue the good work and best of luck.

Devil's advocate: You may find that it better achieves your goal to provide packages and say "For power users testing only". You might also be able to petition Debian to have it removed if it's leading to people foot-gunning.

If I can help with deb/rpm packaging, let me know. I'm a power user here, excited about bcachefs, but haven't tried it because I don't have the time to give it a worthy test (remembering how much time I put into getting ZFS stable on Linux). Thanks for your hard work on it!

I'd love for someone to pick up the deb packaging. My only requirement is no unbundling :)

If we can't get it into the official debian repo like that, we can just ship it as a ppa for now.

It seems honestly odd to me not to just vendor what you need as standalone packages if your dependencies are that specific and you're a filesystem i.e. you use the bcachefs-errno package, not errno.

Debian tends to put their principles above pragmatism (for better or worse), so would they even agree with such vendoring when it's entirely meant as a way to bypass their vision/requirements/process for how dependencies should be handled?

That particular principle is bourne of pragmatism; Debian long ago learned the lesson that other distros are determined to relearn ( https://thenewstack.io/vendoring-why-you-still-have-overlook...) - vendoring is not good for security. In fact, I have come to view Debian's commitment to principles as almost always a practical matter, because those principles (almost?) always trade short-term pain for long-term quality and stability.

It's also effectively what the big cloud vendors do with their monorepos. This makes sense when you have upstreams which are slow at upgrading (e.g. it looks like Debian is upgrading packages packages using older bootstrap to bootstrap v5 across the board, and such fixes get pushed upstream; there's also tooling to watch new releases, so Debian's tooling effectively acts like a system-wide dependabot).

This isn't a Debian problem though, that's the point: if bcachefs-tools has such specific dependencies, then why doesn't it vendor it's own dependencies so it's clear they are packaged and used independently?

A bunch of the packages in the release at hand were actually upgraded, not downgraded, for example.

A counterargument would be that Rust+Cargo pins specific versions already. If you’re writing Rust, you should rarely need to vendor anything unless you’re maintaining a patched version or something.

Vendoring also bypasses the package cache and build cache. If 2 apps depend on foo-1.2.3, they can normally share the cached package and its build artifacts.

Basically, Cargo goes a long way toward ensuring you rarely need to bother with adding someone else’s code to your repo.

Cargo does a per-project build cache, not a shared one.

Oh, guess it does. I've been using sccache so long that I'd forgotten that.

Do you know off-hand why it doesn't, though? If 2 packages use foo 1.2 with the same features and, say, the default build settings, why not share them by default?

I think at the time Cargo was made it was just far simpler to implement. It's not just that, it's also rustc version, sometimes environment variables... much less likely to cause problems by keeping it per project. Of course that stuff still needs to be kept track of, but like, "to get a clean build, kill this directory" seems easier. Not sure if there is an explicit justification written down anywhere from 11 years ago.

Reply on news.ycombinator.com