"Rust is obviously a better choice for Linux than C." "Ok, if it is obvious, it should be easy to demonstrate?" This is not only about Rust - 99% or so of all new tech can't demonstrate any benefits.
If you want empiricism try academic papers instead of mailing lists ;) Some numbers: most critical bugs in the Linux kernel are due to memory safety: 40 out of 65 bugs allowing for code execution found in Linux in 2017 could have been prevented by using a memory-safe language [0]. The paper is about Go, but the same logic applies for Rust. Fun fact: 39 out of these 40 bugs were in drivers [1], so I think starting with drivers in Rust is a good idea.
[0] https://www.usenix.org/conference/osdi18/presentation/cutler (really good read!)
[1] https://arxiv.org/pdf/1909.06344.pdf (disclaimer: I'm a co-author of that paper)
To address the particular point being made here, lots of things in EE are experience driven, for example which decoupling capacitors to use or to not use 90 degree angels on pcbs. I think many aspects of typical engineering are way more driven by experience and vague rules of thumb than we expect looking in from the outside.
The reason why this is more obvious in software development may be because it's pretty hard to have good measurements and understanding failures is pretty hard and working around them is pretty easy.
If anything, software development is sandcastle-building. We all have ideas on how to build it, but at the end of the day we're just amateurs fucking around with plastic shovels.
This. And even then, after the calculations are done the engineer add a “safety margin” (and it's not 10%, more like 500 or 1000%).
It was Microsoft that got people used to software failing on a routine basis, and "have you rebooted?". Before, anything that ever crashed, you sent back for a refund, or expected a tech to come out and fix it post-haste. Clearly MS could not make money that way, but getting people with no experience to believe failure was normal was quite cheap to arrange.
Apart from that, I think you are giving too much credit to construction engineering outside of the highly regulated sectors. They mess up just as often, and expensively so. But stakeholders in construction are slighly more aware that their requirements are going to be set in stone, pun fully intended.
What really makes a difference is the malleability of software. This is an invitation for feature creep and last-minute changes. Also, the impact of overengineered, unmaintainable software is not quite in-ya-face as for physical assets. Therefore, in software engineerin project managers need to be religious about these matters. And software engineers must get better at communicating about these concerns.
Claiming that an entire science field "lacks empiricism" is quite brave, and definitely false in the case of CS.
Perhaps you are talking about software engineering in practice instead, but that is false too (for any serious project).
> "Nevertheless, we believe that, even today, the advantages of using Rust [in the Linux kernel] outweighs the cost." Where is the evidence?
Three sections of the RFC are spent on characterizing the sentence you quote. The LWN article is not the RFC.
Furthermore, many of the claims I wrote in the RFC are simple facts (e.g. features that Rust has) that you can easily corroborate. For other claims, you can look up plenty of articles, scholarly and otherwise, about Rust benefits.
But none of that really matters, because the only way to gather the evidence you are actually requesting (Rust in the Linux kernel) is getting into the kernel and evaluating Rust after a while.
> "Rust is obviously a better choice for Linux than C."
The RFC does not claim it is "obviously" a better choice.
Quite the opposite, in fact: see the "Why not?" section.
> Perhaps you are talking about software engineering in practice instead, but that is false too (for any serious project).
I'm not the first one lamenting the lack of experimentation in CS. See https://www.cs.princeton.edu/~jrex/teaching/spring2005/fft/m..., http://www.inf.fu-berlin.de/inst/ag-se/teaching/S-Komponente..., https://ieeexplore.ieee.org/abstract/document/1027796, https://ieeexplore.ieee.org/abstract/document/4221632
> Three sections of the RFC are spent on characterizing the sentence you quote. The LWN article is not the RFC.
I know, but it doesn't contain (empirical) evidence.
> But none of that really matters, because the only way to gather the evidence you are actually requesting (Rust in the Linux kernel) is getting into the kernel and evaluating Rust after a while.
That's the medical equivalent of giving the patient an unknown drug to see what happens. Did the patient get better? Maybe it was the drug! Did the patient get worse? Maybe it was not the drug!
A better way could be to fork a less-visible (but still well-maintained) C project and rewrite it in Rust. If the defect rate over time is significantly lower than for the upstream C project then that is empirical evidence in support of a Linux rewrite in Rust. Running such an experiment would take a lot of time and effort, but that is no excuse for not doing it.
Take your first IEEE paper's abstract:
"Empirical software engineering research needs research guidelines to improve the research and reporting processes."
They are pointing out problems with the empirical research that is being done, not claiming "almost complete lack of empiricism in Computer Science" as you did.
There are issues in all science fields, their research, their statistics and their reporting, in particular outside their major journals (and not even those are infallible). Calling out an entire science field in particular is quite insulting to all their researchers.
> I know, but it doesn't contain (empirical) evidence.
Empirical evidence of what? Please do not remove/ignore parts of my reply to suit your argument.
You may claim that you would prefer the RFC to include references to papers and articles; but you cannot claim the evidence does not exist ("99% or so of all new tech can't demonstrate any benefits") or that an entire field has no clue ("the almost complete lack of empiricism in Computer Science").
> That's the medical equivalent of giving the patient an unknown drug to see what happens. Did the patient get better? Maybe it was the drug! Did the patient get worse? Maybe it was not the drug!
No, it is not equivalent, because we are not unconditionally adding Rust into the kernel (much less "rewriting it" as you claim). In fact, one is free to have a C and a Rust version of the same driver and compare whatever metrics you need. Even the RFC contains such an example and I am aware of at least one company doing precisely that.
Moreover, Rust is not an "unknown language" in the sense of an "unknown drug". On the contrary: it was explictly designed to address some issues such as memory-safety.
> A better way could be to fork a less-visible (but still well-maintained) C project and rewrite it in Rust. If the defect rate over time is significantly lower than for the upstream C project then that is empirical evidence in support of a Linux rewrite in Rust.
Not really, because 1. you would need to show the same results would apply to the Linux kernel and 2. we are not rewriting the kernel in Rust.
> Running such an experiment would take a lot of time and effort, but that is no excuse for not doing it.
As mentioned, I am aware of at least one company running an even better experiment than the one you propose.
If you want faster results, then you will need to put money into it yourself.
When someone thinks their hashmap is faster in theory they get asked to benchmark it... it often isn't in practice.
But when someone decides that some norm in whitespace, use of parenthesis, or use favored control-flow concepts will result in a lower defect rate or reduced maintenance costs we somehow are expected to accept these conclusions without clear evidence.
Even though everywhere is in computer science where we can easily measure results there are countless examples of theoretically better stuff that works less well in practice. Why should we expect our intuition on human interactions to be any better than our intuitions over which data structures are faster?
But even in medicine, probably the area with the most extensive empiricism (we have insane amounts of data on mouses and humans), there are claims that won’t ever get to the point of doing a study on. There are stupid things that get tested, eg homeopathy (it has plenty of studies showing it is quack), but I think noone would say it is irresponsible for a doctor to claim that hand healing over a gun shot is not effective, even without direct evidence.
Of course there are surprising findings everywhere and as other commenters wrote, especially in low level optimizations the “obviously slower” solution can easily win. But I would wager that C is so terrible as a language with so little protection from even trivial mistypings, no real abstraction power requiring text-based macros that simply should have never existed to begin with, and the worst of all, not even goddamn namespaces making everything’s name indecipherable. On the other hand, I’m not necessarily bought that Rust’s borrow checker is the best way for handling memory-related errors, for example a syntactically better-than-C language like C++, or Zig might reduce memory bugs just as much, we don’t know.