https://lore.kernel.org/rust-for-linux/208e1fc3-cfc3-4a26-98...
The “in hindsight” version of how this should have gone without ego:
* Patch adds Rust bindings to C API
* Maintainer has concerns about increased maintenance cost and clarifies policy about C changes and Rust abstraction if unsure.
* Since R4L is approved, the binding is allowed to exist in a form that doesn’t inhibit changes to C API. C API is allowed to break Rust (I assume, otherwise the entire effort is moot).
* Any concerns about tooling etc which DON’T exhibit this property (and can demonstrably show that merging this Rust code will make C code harder to change) are brought up.
* These are resolved as a tooling issue before the change is merged (I don’t think there are any in this case).
All the discussion about multi-language projects etc is for the project as a whole to decide, which happened when R4L was approved and the breakage policy was decided (might need to be properly formalised).
If the maintainer (or anyone) is unreasonable, then the only approach is to have someone with more authority weigh in and make the decision to bypass their objections or sustain them (which is sort of the direction this was going before the diatribes).
> If the maintainer (or anyone) is unreasonable, then the only approach is to have someone with more authority weigh in and make the decision to bypass their objections or sustain them (which is sort of the direction this was going before the diatribes).
While they were arguing, Linus said nothing. While the maintainer was issuing ultimatums, Linus said nothing. Linus only said something when social media forced his hand. This is the real issue.
IMO, it seems inconsistent to green light R4L and not declare a clear policy for Rust code interacting with C code without adding a hard dependency (and if it WAS declared, not enforcing it).
The only benefit of doubt I can give is that there wasn’t enough time for Linus etc to weigh in before the thread got sidetracked (and the decision became much more politically charged). It’s unclear what would have happened if only the maintainer was unreasonable.
People are wrong in LKML often.
This time, somebody was wrong in a much worse way than usual.
On the other hand, I once ran into an issue with uboot where a bad update knocked out my emmc, usb and sata controllers... found an email address of someone developing the dtb files and got in touch with them, and it was fixed in under a week.
At the end of the day, people are weird sometimes. I wish all the best for marcan.
I think they expect people who want things to advocate harder than just mentioning it once. If no one brings it up again, then they assume that no one cares.
Should probably have just asked again, or sent a small one-line patch. It's "mention something on Slack" vs "creating a GitHub issue/PR"
Have there been any recent popular developments on a similar workflow that is as robust as e-mail ?
I'm not sure how long GitLab can be trusted either, as well as git becoming a bit too synonymous with GitHub...
What? PRs or issues being forgotten happens all the time, especially for large projects.
They did take months to finally land, and the whole process of getting my corp email address to be compatible with their email-based PR system was way more of a faff than it had any right to be, but they did land. You can install mainline Linux on a Legion Go now and the display and controller will behave as expected, out-of-the-box.
I cannot claim to have felt the effects on the maintainer-side of this workflow in large-scale projects though.
Suppose you find a bug in the kernel and come up with a patch. You email the patch to some kernel mailing list and ask for feedback. Typically, you will receive no response whatsoever, because no-one is responsible for responding. You can try emailing random developers and eventually maybe one of them will have mercy on you.
In Firefox and I think Chromium, you can file a bug, attach your patch, request review from someone (the UI will help you choose a suitable person), and it's their job to respond.
In Firefox you have to fiddle with Mercurial, phabricator, and their homegrown CI. In Chromium its Gerrit and their homegrown CI, and oh btw you touched code that lacked tests so tag, you're it.
Firefox and Chromium's bespoke tools have their pluses and minuses but they're a lot easier to deal with that the kernel "workflow".
I still remember the story where some other guys had to meet some Mozilla folks for lunch and nag them for reviews…
In Firefox, in my era at least, a reviewer who simply ignores a review request indefinitely was not doing their job and would get yelled at by someone --- me, if it came to my attention.
I'm grateful the kernel still supports MIPS, which means an old appliance of mine still works perfectly fine and is upgradable. I would be cery sad if someone were to rip-out support of an old MIPS arch, just because it's old and unwieldy
The first few times, it took me longer to figure out how to send a patch than it did to fix the bug I was writing a patch for.
That’s seriously lame.
Harder to understand and contribute is a bad, but unless there is a proposal for a similarly flexible system that has minimal downsides and massive advantages, the preference of existing maintainers should dominate over potential future contributors. Especially factoring in how expensive of a migration it would be.
Linux is so ubiquitous & important that might never happen, maybe it will just become increasingly captured by corporate contributors who can at least build long lasting repos of knowledge and networks of apprenticeship to help onboard newbies. Open source in name only.
it sounds the opposite of optimized to me. Unless we're optimizing for something other than developer experience and efficiency?
It's almost like there are more important goals wrt software development.
Any change introduces anomalies and those cause all sorts of hell.
I won't contest that there are advantages to the linux Kernel's workflow, but there are downsides too, and a major one is that it scares off potential contributors.
That said GitHub definitely is far from perfect as well, and has different strengths and weaknesses from email based flows. As do any other options.
But just because there isn't currently anything that is unilaterally better doesn't mean things can't be improved. There is clearly a problem with onboarding new developers to the linux workflow. That should be acknowledged, and a solution sought. That solution doesn't have to be switching to GitHub or similar. Maybe there just needs to be better documentation on how to set up the necessary tools, that is oriented towards developers used to the Github process. Maybe there needs to be better tooling. Maybe the mailing lists need to be organized better, or have the mailing list automatically add metadata in a standard, machine-readable format to emails. Etc.
obligatory reminder about breaking someone's workflow https://xkcd.com/1172/
Then I talk to the old timers and they act like I just need to get used to it.
And why not?
- Distributing patches via email is more scalable than web hosting. Even GitHub could not host the level of activity happening on the LKML mailing list
- Web hosting has a variety of access and legal problems when working with international teams; email crosses borders more easily
- Email is a decentralized and distributed system that prevents any single company from controlling a project's development infrastructure (release infrastructure is another story, but multiple companies will generally manage their own release process anyway)
Q: What would make you even more happy with Linux? GregKH: If you contribute to it.
Why the hell would I wish that upon myself?
My impression of a few glancing online interactions is that they're both abrasive but marcan is quite unwise in a way that Linus has had beaten out of him
And he's not just abrasive He's a troublemaker. Seriously, code of conduct violation? It was perfectly clear what Hellwig meant by "cancer".
H: I don't want to support multilanguage codebase
R: We'll have a maintainer verify R4L is behaving properly.
H: I solved issues because they were unified.
R: Rust will be mirror of whatever C is, and you're free to break it, R4L will maintain it.
H: No.And to clarify I'm not saying he's right or wrong or acting good or bad. I have however expected R4L to ultimately fall apart because of this exact issue, the maintainers have never been on board with maintaining Rust code and that hasn't changed. While that remains the case the project is going to be stuck at a wall - to the point that if they're confident they can maintain the Rust code themselves they should just fork it and do that. If it works well enough they'll eventually be too popular to ignore with people choosing to write their new modules in Rust instead.
I think the conversation is more about people equating R4L as validation for rust or even themselves.
I agree it's rude, offensive, and hostile, but there are degrees of things and context matters. "You are cancer" would be much worse. I feel we should try and interpret things in good faith and maintain some perspective. For a single word like this: you can just read over it (which is also what the other Rust people did).
Certainly outright removing Hellwig from the Linux project, as Marcan suggested, is bizarrely draconian.
As I argued a few days ago: part of "being nice" is accepting that people aren't perfect and dealing with that – https://news.ycombinator.com/item?id=42940591
He said: “ And I also do not want another maintainer. If you want to make Linux impossible to maintain due to a cross-language codebase do that in your driver so that you have to do it instead of spreading this cancer to core subsystems. (where this cancer explicitly is a cross-language codebase and not rust itself, just to escape the flameware brigade).”
There is an argument that being hyperbolic, dismissive, and maybe a bit rude isn't as bad as some people make it out to be, and that it is a fine way to hash out disagreements and arrive at the best solution - that it is simply a result of the passion of exceptional engineers. There has historically been much of it in kernel development. But it seems that as the background and culture of kernel maintainers has broadened, that a more measured and positive approach to communication is more constructive.
It doesn't seem like marcan exemplifies this very well either. It is a loss for someone so skilled to abandon collaboration on the kernel, and seems like the unfortunate result of someone being dismissive and rude, and someone else taking that harder than is warranted or healthy.
Stange, I think interpreting it your way requires bending the mind. Hellwig clearly used it to describe what he sees at the ill effects of multiple languages in the kernel. It was not used to describe either Rust the language or this specifically this particular submission.
Instead of providing helpful advice like outlining the current situation and suggesting specific improvements (action A, task B, and goal C) to reach the goal, it feels rude and offensive.
The only helpful advice, which they did give, is don't even start doing this because it's fundamentally wrong.
The linux kernel is like a house where everyone is a vegan. Marcan believes that incorporating some meat in the diet is important, and better that being a vegan. He may even be right. But so what? He makes his pitch, the family says that's nice but no thanks. He then demands that they eat this chicken because he wants to live in the house and wants to eat chicken while living in the vegan house?
I don't see how he has any right to what he wants, and I don't see an existing kernel devs refusal to cooperate, or even entertain cooperating, as automatically wrong or unreasonable.
While I think this a dumb metaphor, it's also incorrect in this context. The Linux kernel explicitly supports C and Rust code, and there are very clear parameters to allow for Rust code to be integrated into parts of the kernel.
Or in other words, the decision has already been made to allow meat into the vegan household, and now one maintainer is explicitly blocking a package of meat from entering the building, even though it has already been decided from on high that meat should be allowed in.
This isn't quite accurate, though, because of the unnecessary metaphor thing. Reading the original mailing list chain all the way through and talking about these events directly is completely sufficient here. The patch was reasonable within the parameters set out for the R4L project. The maintainer of this subsystem blocked it explicitly because they disagree with the idea of R4L in general (calling it a cancer).
The question is not whether or not R4L is a good thing or a bad thing - anyone can have their own opinion on that. R4L is part of Linux, and will be for the foreseeable future, until it either clearly demonstrates its use, or clearly demonstrates its failure. The question (at least as regards the "cancer" comment) is whether it is okay for a maintainer to describe another team's work as cancer, and to publicly block it as much as they can.
If they are overstepping, then Linus will make that known. Until then, apparently they are not overstepping.
And he can use that image if it communicates the concept he wants to communicate.
It sounds like a valid image to me to apply to the concept of polyglot.
He is saying that "If there is really no way for a rust driver to exist all by itself without any of the c code having to do anything special to accomodate it, then so be it, I guess rust doesn't fit here after all."
rust devs are saying "you're not even helping a tiny bit!". I am saying, no, they're not, so what? They don't have to. They did not request what rust devs are trying to do.
The concession rust devs got to proceed to attempt to use rust in the kernel at all doesn't promise almost anything beyond "well you can try". It does not promise to facilitate that try at all really.
It was used to describe the Rust for Linux project, as well as any other potential efforts to bring other languages into the kernel, of which there are none. It is clear why someone working on the Rust for Linux project would feel that "this cancer" refers to the project that they are working on.
I'm not trying to pull out pitchforks, I don't want anyone to burn. I just want people to collaborate effectively and be happy, and I think it is empirically clear that calling something that grows/spreads and that you think is bad "cancer" is not useful, and only inflames things. It is not an illuminating metaphor.
No, it is not perfectly clear.
The generous interpretation is that they meant it is "viral", spreading through dependencies across the codebase (which I don't think is technically accurate as long as CONFIG_RUST=n exists).
The less generous way to interpret it is "harmful". The later messages in the thread suggests that this is more likely.
But I'm left guessing here, and I'm willing to give some some benefit of doubt here.
That said, the response to this comment was incredibly bad as well.
https://lore.kernel.org/rust-for-linux/CAHk-=wi=ZmP2=TmHsFSU...
(not taking either side, just interesting to read the reply)
> You think you know better. But the current process works.
Regardless of how badly broken the kernel development process is, Linus and others observe that Linux continues to dominate and conclude that therefore the development process "works". Success in the market blinds the successful vendor to their defects. Sound familiar?
If Linux carries on down this path then eventually something else that has been subject to more evolutionary pressure will displace Linux, and we'll all be telling stories about how Linux got complacent and how it was obvious to everyone they were heading for a fall.
And honestly, with maintainers like Hellwig maybe that's what needs to happen.
Linus was right to reprimand him for the suggestion.
https://web.archive.org/web/20250204162031/https://social.tr...
However it seem that you need to disable js as soon as the content load or it will be overwritten by a 404
"Thinking of literally starting a Linux maintainer hall of shame. Not for public consumption, but to help new kernel contributors know what to expect.
Every experienced kernel submitter has this in their head, maybe it should be finally written down."
Personally, seeing
> Being toxic on the right side of an argument is still toxic, [...]
written unironically, on social media, immediately after that person wrote @marcan
> and if that then causes you to ragequit, because you can't actually deal with the heat you've been dishing out coming back around the corner: fuck off
leaves me feeling more sympathetic to marcan's argument about the kernel being full of toxic attitudes, not less. Maybe public shaming isn't the answer but there's a problem here. Maybe don't make comments like that on social media if you want to criticize people for leaning on social media in kernel disputes.
This seems like a tu quoque fallacy. The feedback is either applicable or not, regardless of who said it. They're absolutely correct that Being toxic on the right side of an argument is still toxic.
Even if there is hypocrisy (whether judged by you personally or someone else), it wouldn't invalidate the point.
Certainly in context, this seems fairly reasonable: https://chaos.social/@sima/113961283260455876
Yeah this isn't about "being civil" or "friendly" and even less about "don't call out". This is about calling out in an ineffective and imprecise way, so that the people actually trying to change things are busy patching up your collateral damage instead of implementing real changes, while all you've achieved is airing your frustration for internet drama points.
When you're that harmful with your calling out, eventually I'm going to be fed up, and you get a live round shot across your bow.
And if that then causes you to ragequit, because you can't actually deal with the heat you've been dishing out coming back around the corner: fuck off.
Or as Dave Airlie put it upthread in this conversation: "Being toxic on the right side of an argument is still toxic, [...]"
So please do call out broken things, do change things, I've been doing it for well over a decade in the linux kernel. But do it in a way that you're not just becoming part of the problem and making it bigger.
---
And this is not the first time something like this has happened with Marcan. He may be tired of the Linux devs, but many of them are also tired with him (including some of the people working on Rust, it seems).
And this is part of a conversation on what went wrong here, not an attempt to rally the troops. You really can't compare it to Marcan's stuff. This kind of (selective) demand for absolute perfection is really not great.
If true, then it sounds like there are some “missing stairs”[1] (the professionally difficult kind, hopefully not the other kind) in Kernel development.