Linux Rust Support
lore.kernel.org
lore.kernel.org
On the other hand I'm slightly suspicious of this movement. Rustaceans are very, very eager to promote the language. It would be interesting to know how much this movement comes from within the Linux kernel community or if it is driven more by the Rust community.
In any case the language seems to have potential of becoming a new mainstream systems programming language in a few years.
The rust community is naturally happy to help out, but I can't even imagine us being able to drive this sort of work...
It has an implication that Linux is something that any enthusiastic group of outsiders could force their changes into. And that there are rust people would create such a group of outsiders.
It has an implication that rust people and linux people are disjoint sets.
It's generally taking this strange adversarial view to a collaborative endeavour. People aren't "rustaceans" or not, it's not a sports team. People are people, they generally program in a wide variety of languages, and use whatever they think is best given a certain context.
I'm not sure if I've fully captured what seems "off" to me about the comment, but hopefully you get the gist.
As for asking questions, that's a clear example of https://rationalwiki.org/wiki/Just_asking_questions - but at the same time the general sentiment seems to be frequently reflected in internet comments. I don't think it's a malicious case of this, just how they chose to express their opinions.
To clarify, I didn't mean suspicious in the sense of expecting malicious behavior, rather than in the sense of not being overly optimistic about this type of progress, because I think it is very hard to incorporate new tooling/languages despite their merits.
In other words: it would be nice to see this being fruitful, because it would possibly a strong, positive signal for programming and programming languages in general, but I'm cautious or "suspicious" still.
When I learned rust I thought it felt more C-like than C++ to be honest. Nowadays I’m pretty happy to be in this c to rust to c loop.
I think you’re exactly right about c++ not offering much benefit to someone like Linus. It’s much simpler than c++. Rust has also informed the way I write C (granted, I’ve also gained more experience along the way, but rust forced me to have more discipline for sure).
I mean, "it seems like our technology should fit this domain, let's see if we can help them use it" isn't so strange as conspiracies go.
But you are right, this is not coming from the Rust community itself. In fact, several independent groups have been thinking of using Rust in one way or another for kernel purposes, as early as... 2013 (!), e.g. from Taesoo Kim:
https://github.com/tsgates/rust.ko/commit/58f0fb8fdf4063f5b24a09cbfeec81bf28a9d81b
At the moment, in the project we have a mixture of people that come from a Rust or kernel background, which has been great. And most of us have used many languages in our work, not just Rust. Personally, most of my career has been around C++ and Python.
That's not my definition of long history! My first Linux kernel was just pre 2.0 and I used make config. Actually I used make config for quite a while before it got out of hand and I started keeping copies of the config to work from instead of doing it from scratch each time I did an upgrade.
I'm not the sharpest tool ...
It's long in comparison to Rust!
Rust is not time served and it needs to grow up. Initial signs are positive. We are at the cooing stage over a pretty child that may either be the Messiah (a Naughty Boy) or decide to throw its poo at its parents and stick to licking windows.
Time will tell.
You make it sound as if those are two non-overlapping sets. I don't think they could pull off this project without intimate knowledge of the Linux kernel.
Or that Linus is going through something very respectable.
Which is good, although I wouldn’t have minded a bit of old school Linus for the anti vaxers.
If you look at Linus's criticism of C++ (e.g., https://lkml.org/lkml/2004/1/20/20), he criticizes a few specific things about the language - exception handling, implicit allocations, complex OO. Rust, as it happens, doesn't do any of that. Errors are passed back via return values (which is what the kernel already does), memory allocations are explicit by convention and "zero-cost abstractions" are favored, and the OO model doesn't have multiple inheritance or anything, just interfaces (which is also what the kernel already does, in C, using structs of function pointers).
So I'm not surprised that he doesn't have similar harsh words for Rust. It solves a lot of the issues with C that have been causing practical problems for the kernel (apart from confusing aliasing rules and confusing overflow rules, there was also the problem a while back where a bunch of != NULL checks got optimized out), and it doesn't have the problems C++ has - in fact it tends to solve those use cases in a way similar to the kernel's existing approach.
For my part, I've been involved a very tiny bit in Linux kernel development for well over a decade because I find systems programming cool, and I've also been involved a very tiny bit in Rust starting around 1.0 because I find systems programming cool. I started this project because I genuinely thought Rust would be a good choice, not because I was deeply involved with the Rust community and I wanted to evangelize it.
(I looked up the IRC logs where Alex and I decided to start working on this, and the context was that Alex had run into some bug with buffer length handling in the kernel, he was considering sending in some patches to improve the C abstractions about this, and since PyCon was the following week I semi-jokingly asked if we should instead spend our sprints time writing support to ship kernel modules in Rust. Which we did.)
Rust does not allocate implicitly when you use Vec or String (or any other container) ? come on..
In C++ you can write code like this:
#include <string>
#include <iostream>
void print(const std::string &s) {
std::cout << s << std::endl;
}
int main() {
print("Hello world!");
}
Compare the equivalent Rust code:
fn print(s: &String) {
println!("{}", s);
}
fn main() {
print("Hello world!");
}
To get this to compile, you have to write something like print(&"Hello world!".to_owned()); , where you specifically instruct an allocation to happen.
The objection is to implicit allocations. Explicit allocations are fine.
Such a warning would be a bug in this specific case as "Hello World!" is under the SSO size though - there is no call to malloc/new in this specific std::string creation.
I fail to see how that is relevant though: the kernel does not use the C standard library - it's not going to use the standard C++ library either except maybe the freestanding, compile-time-only bits such as type_traits, concepts, ...
In which case it is trivial to have a string type akin to rust's:
class kstring {
public:
explicit kstring(const char* str, std::size_t len);
private:
const char* m_storage{};
std::size_t m_len{};
};
auto operator""_to_owned(const char* str, std::size_t len) {
return kstring{str, len};
}
void print(const kstring& str)
{
// ...
}
int main() {
// fails:
// print("foo");
// ok
print("foo"_to_owned);
}
> If you code-review "main", it doesn't look like you're doing an allocation because you're passing a const char *; if you code-review "print", it doesn't look like you're doing an allocation because you're accepting a std::string, which has already been allocated.
My experience really disagrees with this part, every time I've been involved in code reviews, it was common knowledge that a function taking a const& or && can cause an allocation at the call site. It's barely a beginner mistake to not take it into account.
The new() functions for these types are const (ie they can be evaluated at compile time) and produce an "empty" placeholder.
Because Rust's Strings don't have NUL termination, the empty string doesn't need any backing storage, likewise for an empty vector.
When (and more importantly in many programs if) you actually add something to the string or vector that may prompt an allocation, but you called an explicit method to do that, so it oughtn't to come as any surprise.
Rust got luckier here, by the time Rust was invented programmers knew they really care about this, and so Rust explicitly distinguishes Copy (this object is just bits, so you can duplicate it by copying the bits), from Clone (this object can be duplicated, but you need to call its clone() method to explicitly do that work) and gives Move semantics to all assignments of types which do not have Copy.
So in Rust "let a = b" definitely isn't doing memory allocation, it just moves bits. If you misunderstand the situation and assumed b is still valid because it was Copy, but actually it isn't, the compiler has your back and will politely refuse to let you use variable b after it was Moved. If you did still need b you can write "let a = b.clone()" and now they both still exist but anybody can see this code might be doing something expensive 'cos it calls this clone method. And if the compiler says "Ahem, you can't Clone that" then you've just learned something very important about the thing that a minute ago you assumed was just a few trivially duplicated bits.
In such a project where allocations matter one would not stare at class definitions, one would ensure that there are no allocations by forbidding use of libraries like the STL and for example disabling operator new and the copy constructor. If the project is safety-critical there are tools which can enforce such rules.
Edit: I've clarified that I was referring to C++ libraries only.
The problem isn't "Kernel code should not allocate" but "Kernel code should not implicitly allocate".
And C++ thinks this is none of your business. You can have no allocation by not providing an allocator - but you can't have no implicit allocation because the language will go out of its way to hide some of the obvious places allocation might happen rather than expose its warts.
Here's my list:
* RTTI: would be disabled
* Exceptions: would be disabled
* STL, stdlib: would not be used
I think you could address this point by using a slightly forked version of a C++ standard library that marked all its allocating constructors and allocating conversion functions with the "explicit" keyword. (But you wouldn't be able to upstream this, ever.)
I'm not sure if there's any other case where you'd have implicit allocations that you wouldn't be able to avoid. You may be right that this is doable entirely with library changes provided you avoid RTTI and exceptions.
One specific weird case is the copy done when capturing things into a lambda: https://godbolt.org/z/Y7fG5hjqq I expect the first allocation here because I'm explicitly calling the constructor, and that's fine, but the second one surprises me a bit, because the copy constructor is marked "explicit". Maybe people who work with C++ more regularly than me find this less surprising.
But that goes back to the problem 'tialaramex raised, about how "How many allocations does this do?" is a quiz question. Distinguishing zero from at-least-one is important, and you can do that by avoiding any heap-allocated-container types entirely, but so is distinguishing one from more-than-one.
Now of course you could say, "Oh I never use objects, those are awful" but then you ought to ask yourself what exactly the point was of having C++ at all. Or you could say, "We'll ban constructors from allocating" but C++ won't help you enforce that rule and the resulting code won't be idiomatic C++. Implied allocation is just normal in C++.
Also note that although Rust's full standard library won't be available in the kernel, you do get Vectors, you get Rust's reference counted "smart pointer" types, Iterators, Option and Result and so on. In Rust such things don't have implied allocation, only explicit allocation which is fine.
Rust used today in the real world does have some places with implied allocation, a notable example is String concatenation. Rust won't let you write
let hw = "Hello, " + "world";
let h = "Hello".to_owned(); // Now it's a String
let hw = h + ", world";
(2) seems the most tricky, but in reality, in a context like the Linux kernel where the libraries would be written from scratch or wrap existing C code, this can be controlled well. If one is including a new language in the kernel, it should be comparably low effort for example to write a custom vector container which exactly fulfills their requirements.
In the end I realize that C (and I assume Rust) makes it easier to reason about allocations in simple pieces of code which don't call functions, but this is not a deal breaker for C++ either.
So I don't think that allocations is the answer to the question of "Why Rust over C++?", as you claimed when you said "The allocations thing in particular". It's perhaps part of the answer, together with other technical reasons and personal likes or dislikes. And I believe that the latter are an important part of that answer that few if any are bringing up in this thread.
Remember, this is the guy who used tarballs as a version control system up until Bitkeeper gave him free (as in beer) licenses to their DVCS. He didn't hate all version control, as much as he thought that was the case - he hated badly implemented version control, which was the standard at the time with things like CVS. That's why he ultimately wound up writing Git once the Bitkeeper deal became untenable.
In this set of old e-mails from a decade and change ago[0] he specifically calls out STL/Boost being unstable and difficult to work with, C++ providing certain abstractions that have undocumented performance problems, exception handling, and hidden allocations.
Of that list, I can think of several things Rust specifically solves relative to C++.
- Rust traits make reasoning about generic code way easier than C++ templates. In C++, you just write your template, and then the template is resolved at the site of instantiation. If it's not a valid type to put in there, then the compiler tells you that you used the template wrong, but it's still your responsibility to figure out why. In Rust, however, generic types have to be constrained with traits; which list out exactly what a generic type requires. So if you misuse a generic type, the compiler can just say 'well, you need Add on parameter T here to call this', and the error will always be on the type you used, rather than some dependent generic type.
- One of Rust's core goals is "zero-cost abstractions". It does this very well, but it also has some non-zero-cost abstractions. These I'd categorize as "transparent-and-predictable-cost abstractions". For example: Rust is known for it's borrow-checker, which provides compile-time verification of memory safety. However, these might not be the most flexible memory-management solution; so it also provides shared-ownership through reference counting. This does have an associated cost, but it's easy to reason about and clearly stated. They even provided both atomic and non-atomic reference counting options because the latter has less cost associated with it.
- C++ uses an exception handling model built around stack unwinding, which requires that all code maintain a valid stack that the C++ runtime can unwind at all times. This can pose problems for kernels which almost certainly will be doing weird things with the stack that the compiler won't entirely understand. While Rust has exceptions (called panics) and supports C++ style unwinding, it also allows you to change the exception handling model to just terminating the program instead, which matches better with how the Linux kernel handles exceptions.
- Rust also heavily discourages the use of exceptions relative to C++. Panicking is mostly reserved for error conditions that are actually exceptional (i.e. unexpected), while error conditions that are expected can be signaled using return types like Option and Result. There's even special language syntax (called Try, or the ? operator) which will automatically return if a value signals an error.
The only thing I can't say Rust solves is hidden allocations, but it seems like the plan Linus has picked is to tame that beast with a Linux-specific core library, with as much as possible mapped to Linux's existing C abstractions. This probably will work out as this is the intended way to extend existing native code with Rust libraries. (It's how Mozilla integrated things like Servo into Firefox.)
It's my understanding that Rust got that term from the C++ community.
* Point one is a convenience/ease of use topic. Absent C++20 concepts or alternatively static asserts and enable_if, error messages will be harder to parse by the programmer. The one place where this may be relevant is for custom containers, but a kernel list for example wouldn't have to implement iterators and certainly doesn't need to follow the STL design which can cause hard to understand errors.
* I don't understand what you mean when you say that Rust "solves" zero-cost abstractions, but C++ has had this goal for a very long time.
* Exceptions would be disabled when doing kernel development. That being said, having exceptions in application development can be very convenient. C++ supports both, but std::optional and the like are not as elegant as in Rust.
By default, just like Ada, C++ allows to disable exceptions and use other error checking mechanisms.
As a real life example, macOS drivers don't use exceptions.
So it's not surprising to me that Torvalds is critical of languages that don't cater to his use case. Nor why he's less vocal about Rust. The allure of Rust is inbuilt support for multithreading, protection against data races, and an efficient memory management system.
Any feature useful for coding libraries is equally useful for kernel coding, because the overwhelming majority of kernel code is identical to what goes in libraries, and much of it is, explicitly, library code.
The landscape has now changed and Rust has now positioned itself as a language that actually does have very key advantages (in correctness and safety). While C++ has made large strides, its advantages now lie elsewhere (eg; drawing on a wealth of performant libraries-- something that the kernel has less of a sue of).
Looking at the complexity of Rust versus C and the very limited benefits one would get out of adding a few drops of memory safety into a sea of C, one has to conclude that technical merit alone does not explain this decision.
In any case, I don't think Linus will write another language rant after he had an epiphany about his "unprofessional" attitude. And if there's enough corporate support combined with some grassroots support, he's probably figured out that he'd lose that battle even if he were nice about it.
I'm not even sure what this means. What do you consider a militant attitude in this context? From what I've seen, most Rust evangelists (at least here) try to explain its benefits while acknowledging cases where it may not provide something better and is instead equivalent but different or slightly worse (it's not like any language can be better in every respect).
> the very limited benefits one would get out of adding a few drops of memory safety into a sea of C, one has to conclude that technical merit alone does not explain this decision.
I don't think that's a valid conclusion, but even if it is, you haven't explained it well enough. You haven't addressed how much more complex Rust is than C no how limited the benefits would be. You've provided a tiny bit more information than "it's not worth it", which I think is far from sufficient to support that position in a discussion that can have as much depth as is easily achieved here.
> In any case, I don't think Linus will write another language rant after he had an epiphany about his "unprofessional" attitude.
It's not like he's stopped calling being stupid for acting stupid or refrained from criticizing technologies he thinks deserves it. There are recent cases (mentioned in this discussion) that outline recent cases where he's done so. I'm not sure why you would assume he's all of a sudden deciding to keep his mouth shut about what would be valid concerns to bring forth if he had them, nothing to do with interpersonal communication problems he's had in the past.
Regarding technical merit, I like to think of Rust as a C++ with switches toggled in different ways and a borrow checker on top. It's complex to think in and to write in compared to C, but it's at the end of the day much easier to write correct code in. I won't go into all the details since that would take a long time, but I hope the kernel team will write about their difficulties with Rust once they encounter them, since that should be a very fun read.
Regarding Linus, I wanted to confirm to OP that their suspicions are not unfounded, because the other answers seem a bit like gaslighting to me, especially the one claiming it's a "weird question". There is definitely more to this than technical merit, but it doesn't have to be anything ill-natured from the Rust community. My theory is that Linus' spirit has been probably broken by the corporations controlling Linux development through their many commits, his getting older and grassroots support for Rust.
I wouldn't trust Big tech as far as I can throw one of their server farms. That said, I doubt that Linus's spirit is broken and he's simply surrendering to them. Assuming you aren't just ~~trolling~~ tin foil hat farming, I'll answer what I think:
I remember Linus said some years ago (probably ~2019 or 2018) that Rust or something like it in the Linux kernel is inevitable. His reasoning was that the Linux kernel is unlikely to remain maintainable in the long run. Due to the C code in the kernel being very non-standard and due to there being a lot of code in the kernel that got written the way it was for a specific reason and most of it being just unwritten implicit knowledge of the old hands. This makes it very hard to onboard new maintainers and PR's often require significant fixes and significant internal expertise to recognize the need for those fixes and be able to make them. As the old hands grow older, they may not be replaceable.
Furthermore, new bugs are now being discovered in the kernel faster than they can be fixed [1].
If I were to guess, Linus is just hoping that Rust or some other modern safe systems language suitable for kernel development (of which Rust is by far the most mature) could help sustain the kernel's maintainability.
As for why Big tech wants Rust in the kernel. I guess that's because they want total world domination. Which will probably be hard to achieve if any idiot can easily zero-day the kernel their infra all depends on and steal all their secret big data which is "the oil of the age of AI" or rootkit a phone making it harder to surveil it's ~~owner~~ addict. ;-)
> Regarding technical merit ...
Rust is very fundamentally different from "C++ with switches toggled in different ways and a borrow checker on top". I could call Java "Cobol with switches toggled in different ways and inheritance on top" but that wouldn't make them quite that similar as I've made them sound. If you think that Rust isn't fundamentally something drastically different from C++, you either don't have a clue about C++ or don't have a clue about Rust. Rust is holistically designed to be safe, which is the only possible way to actually make a safe language. C++, however modern and with whatever programming best practices and tools, is still fundamentally completely unsafe. It helps reduce security vulnerabilities from very very very unacceptably high to merely very very unacceptably high but in this case better is nowhere near good enough. Rust actually eliminates entire classes of security vulnerabilities (with some manageable caveats).
Another difference is that by the time you've addressed all the reasons typical C++ code is unacceptable in a kernel, you are back at simply C. But you can have high level abstractions and safety and, etc. in Rust inside the Linux kernel. For just one difference, Rust's zero-cost abstractions are actually, you know, zero cost. Unlike C++'s supposedly zero-cost abstractions which have unacceptable (in the Linux kernel) overhead. (For example a move in C++ is potentially very expensive, but at worst it is a memcpy in Rust.) But there are plenty of other reasons mentioned in this thread.
[1]https://lore.kernel.org/dri-devel/20200710103910.GD1203263@k...
I'm skeptical that introducing a new language, whatever that may be, would help. In the short term it will make things harder, since maintainers have to juggle two languages and in the long term, as long as the new language does not displace the old one the situation would not improve.
Rust heads into the kernel? - https://news.ycombinator.com/item?id=27248059 - May 2021 (179 comments)
Linus Torvalds on Rust support in kernel - https://news.ycombinator.com/item?id=26831841 - April 2021 (290 comments)
An RFC that adds support for Rust to the Linux kernel - https://news.ycombinator.com/item?id=26812047 - April 2021 (261 comments)
Supporting Linux kernel development in Rust - https://news.ycombinator.com/item?id=24334731 - Aug 2020 (354 comments)
Linux kernel in-tree Rust support - https://news.ycombinator.com/item?id=23800201 - July 2020 (491 comments)
Linux kernel drivers in Rust might become an option in the future - https://news.ycombinator.com/item?id=20833639 - Aug 2019 (254 comments)
With the initial announcement I mentioned that Linux will need quite a customized Rust experience, and was hopeful that this effort could improve the situation for everyone.
It's great to see this actually happen, with changes already flowing back into core and alloc.
A lot of progress in just a few months!
My small contribution to this, https://github.com/rust-lang/rust/pull/84266, is supposed to be just that!
- C pitfalls, probably documented all over the place.
- C idioms. These are hard to locate for most languages; I wouldn't know where to look in case of C.
- Kernel-specific C idioms. This is probably easy to locate and well documented, but I'd still expect it to be several thousand words.
- Code formatting tools. Are there any canonical formatting tools for C, or is it fragmented like for most older languages?
- How to test C code, which IIRC requires jumping through quite a few hoops.
- How to deal with text, which is a nightmare outside of ASCII in most languages.
1. Rust has pitfalls
2. Rust has idioms
3. Rust in kernel would have specific idioms
4. clang-format and uncrustify are two solid choices
5. Nothing is special about C that would make testing hard.
2. You are cherry picking. He also said "These are hard to locate for most languages; I wouldn't know where to look in case of C." Rust has some advantages from for example Clippy and from a very newcomer-welcoming and helpful community.
3. Rust in kernel would indeed have kernel-specific idioms. But it'll probably be very similar to other embedded Rust and relatively easy for a newcomer to pick up. (Ee'll have to wait and see, but there is good reason to expect this.) C in kernel is known to be very hard for other C developers to pick up.
4. Thanks for the info.
5. But some things are special about Rust that makes testing easy.
6. The kernel doesn't deal much with text and even when it does, it generally doesn't need to deal with encodings (let alone modern encodings). That said, text is really a nightmare in C. Even just ASCII is needlessly complex due to null-terination, badly designed functions and the regular C UB pitfalls. At least you did acknowledge it may indeed be a problem for C compared to Rust.
We have been in contact with a set of companies and academia members
that would like to use Rust as a second language in the kernel. Some
of them have already started to evaluate Rust for their needs using
the infrastructure we have already in place.
In particular, we have got a few statements from major companies.
In no particular order:
Microsoft's Linux Systems Group is interested in contributing to
getting Rust into Linux kernel. Hopefully we will be able to submit
select Hyper-V drivers written in Rust in the coming months.
Arm recognises the Rust value proposition and is actively working
with the Rust community to improve Rust for Arm based systems.
A good example is Arm’s RFC contribution to the Rust language which
made Linux on 64-bit Arm systems a Tier-1 Rust supported platform.
Rustaceans at Arm are excited about the Rust for Linux initiative
and look forward to assisting in this effort.
Google supports and contributes directly to the Rust for Linux
project. Our Android team is evaluating a new Binder implementation
and considering other drivers where Rust could be adopted.
In addition, IBM contributed the Rust kernel support for PowerPC
which was already included in the RFC.Well [0]
https://os.mbed.com/docs/mbed-linux-os/deprecated-product/we...
I've never read anything to make me think that Rust is a safer alternative than any of these options; the guarantees of the language are just assurances of its advocates.
- the requirements generally change quite frequently
- the requirements are generally a long way from being "formally verifiable"
Where I, personally, have found the most value so far using FV techniques is from modelling "tricky" things whose requirements are quite solid. A few examples from my work:
- A state machine for managing LoRa communication. The requirements in the LoRa spec are pretty good. I took those requirements and designed a state machine I could implement on a low-power device and modelled that SM in TLA+ to verify that there were no deadlocks/attempted transitions to invalid states.
- A state machine for managing the Bluetooth communication between an Android app, a Bluetooth-Iridium (satellite) bridge, and the server on the other side of the Iridium link. Similar to the LoRa case, I used TLA+ to ensure that there were no deadlocks in my proposed SM (and uhh made a number of corrections as I went). Ultimately, due to the design of the BT/Iridium gateway, the exercise resulted in the conclusion that it wasn't actually possible for this process to be 100% reliable but it was possible to detect and retry the edge cases.
- Modelling a somewhat awkward email/SMS invite flow for a mobile app. You could get an invitation over email or SMS, and your account transitioned through a few different states as both your email address and your phone number were verified. Modelling this helped ensure that your account couldn't get into a state where no forward progress was possible or into a state where you could use a half-completed account.
Just a note, I (personally) don’t consider TLA+ a FV technique — it is a really good technique that actually scales much better than formal verification programs like Agda or Coq.
Is TLA+ a sandwich?
I think CSIRO disbanding the team was a huge mistake, but in any case they weren't producing the kind of success metrics that sustained their funding.
From a quick google, seL4 has around 10,000 lines of C code, whereas the Linux kernel has around 30,000,000.
Let us see what they have, using formal verification (Ada/SPARK).
- JWX is a library for handling JSON data and more. It is implemented in the SPARK programming language and has been proven to contain no runtime errors. As a result, JWX is particularly suited for processing untrusted information.
- The RecordFlux specification language is a domain-specific language to formally specify message formats of existing real-world binary protocols. Its syntax is inspired by Ada. A detailed description of the language elements can be found in the Language Reference.
- Formally verified, bounded-stack XML library
This is just Componolit.
Let us check the F* language:
- F* (pronounced F star) is a general-purpose functional programming language with effects aimed at program verification. It puts together the automation of an SMT-backed deductive verification tool with the expressive power of a proof assistant based on dependent types. After verification, F* programs can be extracted to efficient OCaml, F#, C, WASM, or ASM code. This enables verifying the functional correctness and security of realistic applications. The main ongoing use case of F* is building a verified, drop-in replacement for the whole HTTPS stack in Project Everest. This includes verified implementations of TLS 1.2 and 1.3 and of the underlying cryptographic primitives.
---
Given the above, what exactly do you mean that it does not scale?
Features:
* AES-128, AES-192, AES-256
* AES-CBC (all supported AES modes)
* SHA-1
* HMAC-SHA1
* SHA-256, SHA-384, SHA-512
* HMAC-SHA-256, HMAC-SHA-384, HMAC-SHA-512
* PRF-HMAC-SHA-256, PRF-HMAC-SHA-384, PRF-HMAC-SHA-512
* RIPEMD-160
* HMAC-RIPEMD-160
* ECDSA, ECGDSA
It is pretty awesome. :)
For a less hypothetical example. RedoxOS has a codebase that dwarfs Sel4 (but it is still much smaller than Linux in turn), yet it type checks with much less compute cost than it takes to formally verify Sel4. Or heck, Redleaf actually formally verifies in a tiny fraction of the time it takes to formally verify Sel4, because the Rust type checker not only significantly reduces the amount of requirements that still needs to be formally verified but also reduces the complexity to verify those requirements to linear to the codebase size due to the restrictions that the Rust type system and module system puts on valid Rust programs (with a little bit of care when writing the code). Of course, until you have a certified Rust compiler there might still be miscompiles of Redleaf and until all the features of Rust used in the Linux kernel is formally specified, we cannot do the same with the Linux kernel as for Redleaf even if somehow all of Linux got rewritten in Rust. But in Rust at least it is conceivably possible given sufficient resources and sufficient time. I very much doubt it will ever be possible without help from a sufficiently expressive and practical type system.
Oh yea. Then there's another problem. Most (all?) of the examples you've mentioned have requirements that are well specified and pretty much set in stone. The requirements fro the Linux kernel... aren't quite as obliging.
It's security parameters are well-defined enough that you can use it to launch isolated VMs with... less secure OSes that are sandboxed from each other.
FWIW I would trust anything language-level over applying static analysis tools to C (for an evolving codebase like Linux, having the safety be a first-class part of the codebase is a must) or extracting C from another language (the semantics of C are surprisingly weak, such an extraction can only ever be as good as your model of C which will probably be buggy).
What Rust focuses on is providing a scalable approach to a limited kind of formal verification - focusing purely on type- and memory-safety, not conformance to arbitrary specifications. The latter is of course being worked on, but type safety and memory safety are table stakes.
What are types if not arbitrary specifications? Are you suggesting that formally verified software is not memory safe? What on earth does "scalable" even mean? A verified program cannot scale? If not, why not?
I believe the comment that types are aligned with the syntax is meant as a contrast to other languages which include specifications written in a format substantially different from or fully removed from the implementation. This can reduce the friction of using type-based verification when compared to formal verification capable of describing an arbitrarily complicated spec.
When discussing the scalability of types, I think he is saying that because types are coupled to the implementation, and don't support non-local reasoning, it is less likely that you will run into issues with type checking as you try to compose many small components into a larger program. In contrast, my impression is that with full formal verification, it can become extremely difficult to properly verify a large system.
Regarding your comparison to C and python, I think it's clear that the parent was comparing the specific type system and borrow checker that Rust provides vs. formal verification, not making a statement about the general concept of a type system. In particular, I don't think it's reasonable to assume he was saying the existence of types provides any sort of safety. Rather, it's clear he was saying the use of a powerful type system (such as that found in Rust or Haskell) to implement a limited specification of the program functionality can provide a degree of safety.
"In other words, the Curry–Howard correspondence is the observation that two families of seemingly unrelated formalisms—namely, the proof systems on one hand, and the models of computation on the other—are in fact the same kind of mathematical objects.
If one abstracts on the peculiarities of either formalism, the following generalization arises: a proof is a program, and the formula it proves is the type for the program."
I'm assuming "scalable" in this case is referring to the fact that fully verifying a piece of software is currently takes a large amount of effort. Rust's approach is more "scalable" in the sense that they decided to focus primarily on memory safety, and punted on other forms of verification in the hope that it would be more ergonomic/require less effort.
For instance, I have to deal with JSON. There's no reason older languages can't deal with JSON, but it'll often be inconvenient as the libraries try to deal with mismatches in the type system or whatever, and it just can't ever be quite as slick as a more modern language written with some partial, vague combination of the fact that JSON exists in mind, together with the sorts of techniques used to implement JSON support in modern languages. Or sometimes it'll just be the dominant library for dealing with it just has an old design that baked bad assumptions in, and since it's the dominant library it's the one everything uses, so I have to deal with that everywhere.
And it's just the death of a thousand of those cuts. "But you can do JSON in my favored language!" Yeah, but it's just not quite as slick. "But we have an HTTP client in this language!" Yeah, but it's just not quite as slick. "But we have GPU support in my favorite language!" Yeah, but it's just not quite as slick. And you've ground all that "not quite as slick" deeply into your language over the decades.
So nobody has any one reason they can point at as to why Ada isn't going come back, or why E isn't going to come back, or why Cyclone isn't going to come back... but the reality is that the sum total of the issues is more than sufficient to prevent it from ever happening, and what's going to happen is a more modern language is going to get the support.
Advocates also try to "defeat in detail", in analogy to the military technique, arguments to the contrary, but the problem is that while that may satisfy the advocate, everyone else is still left looking at a metaphorical battlefield that has an exhausting number of "battles" over basic deficiencies, and being "defeated in detail" by an advocate just transfers to exhaustion with the underlying language anyhow.
It probably isn't a "the issue" with Ada/SPARK. It's a horde of small little things, the sum total of which is too much hassle for anyone to want to deal with.
I’m also not sure why “nice JSON libraries” is a hard requirement for kernel code. Also, see JWX.
What does happen with such languages is that someone comes along and creates some sort of new language taking the best of the old language and mixing it with more modern stuff. I personally think there's an Ada-like niche waiting to be exploited there, probably something that also takes some lessons from Haskell. (Haskell is still alive, but IMHO, headed toward this sort of dead right now, and really could use this sort of revival itself.) An example of this is Elixir... Erlang would probably be also something I'd be calling "dead" if it weren't for Elixir.
(Ironically, often the biggest partisans of the base language are the last to know about the revival language because every deviation from the base language is seen as a flaw from the One True Answer....)
Too long of this pattern and it becomes much more trouble than it's worth to use on a business level.
They gave a bunch of reasons.
Now, I know that there is a trendy view of "the more the better", and it does not have as many libraries there, but that has to do with the fact that it is not yet widespread among Ada users. Alire is really new.
> That never happens. It may well klunk along for more decades... but it's dead
Ada is used in 2021 in the exact same places that Ada was used in 2011 (and 2001 before it), if not more. Ada was designed for a specific purpose, one that it still fulfills very well. People have been claiming Ada was dead for decades, yet still it miraculously lives on. Despite Rust, despite Haskell, despite an algal bloom of "formally verifiable subsets of C", and so on.
> What does happen with such languages is that someone comes along and...
Yet... This just... hasn't happened. A successor language more suitable for safety-critical development has just simply not appeared. The manufacturers using Ada —a subset of which can be seen at this[3] link— ostensibly don't seem convinced that any of Ada's many alternatives offer a more appropriate solution for developing safety-critical software.
> I personally think there's an Ada-like niche waiting to be exploited...
The Ada-like niche you describe is, believe it or not, currently occupied by Ada. Ridiculous as this may sound, Ada is actually pretty good at fulfilling its intended function. What needs to change? Ada is not designed to fill the role of Node.js, Java, or C#. It's designed for safety-critical embedded systems, which is where you're most likely to find it today.
> Ironically, often the biggest partisans of the base language are the last to know about the revival language because every deviation from the base language is seen as a flaw from the One True Answer...
Does this apply to your thinking too, or just the proponents of Ada, and other languages you don't like? Are the Rust users just blind to the benefits of Ada? Are you susceptible to the same cognitive blind-spot you accuse us of having?
[1] https://www.npmjs.com/package/pad-left
Wait! There is Alire[1][2]! There is a list of "crates", you can easily include them into your project, and so forth. This is pretty much the same as https://crates.io or https://pkg.go.dev, which means people really should throw their misconceptions away and actually do some research before concluding that the language (Ada) is dead. A couple of quick searches quickly tells me that Ada is pretty much alive, with all of these "modern libraries with modern solutions", and full-on safety without performance loss.
This comment is intended for the person you replied to. :)
[2] https://blog.adacore.com/first-beta-release-of-alire-the-pac...
[0]: https://www.fstar-lang.org
[1]: https://www.microsoft.com/en-us/research/project/checked-c/
[2]: https://www.microsoft.com/en-us/research/project/vcc-a-verif...
[3]: https://github.com/koka-lang/koka
[4]: https://github.com/dafny-lang/dafny
[5]: https://www.microsoft.com/en-us/research/project/project-ver...
I know Rust; I don't know Frama-C or Idris or Ada. What little I know of Ada indicates that it doesn't actually solve the problem because it doesn't have safe dynamic memory allocation. Ada advocates have told me I'm wrong and there's some new thing (inspired by Rust) that does this. Maybe! I'm not sure there's any benefit to the world in arguing it; someone skilled in Ada should just write something and show how it would actually work.
Write a kernel module with any of those techniques, send in a patch, and see what people say. Any effective means of making the kernel less buggy is great. It doesn't have to be Rust.
I claim that Rust is effective because, well, those patches are there, and frankly I haven't been doing any work for this project in almost a year and we're getting so many enthusiastic contributors. So, apparently, a lot of other people also know or are willing to learn Rust. Maybe it's actually better than the alternatives you mention; maybe it just has better documentation. I don't know. I'm not claiming Rust is perfect, just that it's empirically likely to solve the problem.
If you can get together a community of folks who can do any of these other approaches, that would be fantastic. Try it!
It does, just not the way most are used to it.
Surely if one wants to code like malloc()/free(), free() is unsafe in Ada and requires the Ada version of unsafe {}
However many type constructions in Ada do dynamic allocation by themselves, including on the stack.
If memory allocation fails, an exception is thrown and a retry with less memory requirements is a possible execution path.
Ada also has RAII since Ada95 (controlled types), and yes SPARK formal specification now allows for ownership rules.
So, since we're talking about the Linux kernel, that's a red flag straight away. Notice a whole section of this new LKML post is about their implementation of Rust's alloc crate? Linus doesn't want useless exceptions here and has previously objected on that basis.
Meanwhile... Over in the C++ world they have this same behaviour. For them these exception handlers are a fiction, in reality popular standard libraries (never mind everything else you're using) will blow up in low memory and it's game over. Maybe Ada does a lot better. Maybe.
Ada also has no exception allowed profiles and other mechanisms.
HN is not the place to hold a lecture on Ada features.
It is not a maybe, rather lack of knowledge.
We can lead academic discussions about which language is safer/better/leaner/whatever-er, but the fact is that Rust has a larger and growing mindshare so this effort is unsurprising. For Ada, the solution isn't to try and prove to anonymous forum goers it would've been better choice if only people recognized the fact. The solution is to mobilize the Ada community to put the same amount of effort into the publicity and marketing (yes, the bad M word) of their ecosystem.
Ada clearly lives on in many safety critical systems, but judging by the admittedly biased topic frequency on HN and other technical sites, it has a tiny community who mostly keep to themselves. That's fine, but then don't be surprised when it's Hip Language X that gets the headlines and these sort of efforts.
My only point is how many in the Rust community advocate as if Rust would be the first language having features, that actually were already in Ada for the past decades.
As for Ada on Linux, naturally Linus would never go along it.
A C kernel guy liking a language with Pascal syntax, a community that worships a paper that hasn't been true in the Pascal world since early 80's, not even at gunpoint.
If you want to see a use case where Ada was chosen against Rust, check NVidia's project for self driving vehicles firmware.
Plenty of safety conscious language have had followings since ALGOL, this is the point that many miss when praising Rust, while ignoring our computing history.
I think that is a false perception created by the dev-blogging community. Based on GitHub activity (https://madnight.github.io/githut/#/pull_requests/2021/2) Rust usage is declining (absolute numbers show little growth). Based on developer surveys (https://www.jetbrains.com/lp/devecosystem-2020/) very few people use rust outside of hobby development.
I think the concern is that the core argument that "I think Linux should adopt rust for safety" is a bit dishonest, given that there are a large number of well-vetted alternatives. "I think Linux should adopt rust because I like rust for social reasons" would probably be more correct. That Google staff seem to think rust is good for the Linux kernel but not, say, Zircon, would be surprising if those same people actually believed that any legal rust program is safe and inherently trustworthy. "Embrace, extend, extinguish" wasn't that long ago.
I mean, neither Ada nor Zircon are even on the lists, so in terms of mindshare, they are clearly blown out of the water by Rust (from the evidence you provide). To discount Rust based on usage when the alternatives being suggested don't even make the rankings is also a bit dishonest.
In fact, the argument "I think Linux should adopt rust for safety" makes perfect sense. Here's finally a safety conscious language that seems to be embraced by a larger community, and even the big tech players. Instead we get people suggesting arcane alternatives that may be technically better, but have a snowball's chance in hell to become commonplace any time soon.
In the meantime, Google makes extensive use of Rust in new components of both Fuchsia and Android and even rewrites some legacy components where it makes sense.
I think you and 'grumblenum should team up and write a working kernel module in Ada/Spark and then we can have a fruitful discussion about it, instead of just theorizing about whether it could be done. I'm interested in seeing the problems solved, and I'm indifferent to technology. If you think Ada/Spark can solve them, that would be awesome!
(Editing to reply to a small claim in another subthread, "Linus would never go for it" - we believed the same thing when we started. We were originally thinking that we'd pitch this directly at Linux distros shipping out-of-tree modules that they didn't want to keep having to push security updates for. https://github.com/fishinabarrel/linux-kernel-module-rust was out-of-tree and we tried to keep compatibility with kernels 4.4 onwards. But it turned out Linus isn't opposed to it.)
I think the first problem is some kind of language specification. That it's not possible to know what behavior one should expect from rustc means that it isn't a solution. It's another problem. You should check out Ken Thompson's great speech on why this is an issue. https://www.cs.cmu.edu/~rdriley/487/papers/Thompson_1984_Ref...
Or do you think it's wrong to regard unauditable enhancements to free software paid for by Google with suspicion?
Here's a way to address the issue he brings up in that speech that does not involve language specifications: https://dwheeler.com/trusting-trust/
https://www.adacore.com/academia/projects/muen-project
https://www.adacore.com/company/partners/nvidia
https://genode.org/about/road-map
https://www.ptc.com/-/media/Files/PDFs/Developer-Tools/apexa... (ApexAda Exec)
https://www.ghs.com/products/safety_critical/integrity_178_e...
https://www.windriver.com/products/helix (Wind River Helix Bare Metal partitions)
Muen: If I am reading the allocator in https://git.codelabs.ch/?p=muen.git;a=tree;f=tools/liballoc/... correctly, it simply has no deallocator function at all. It does have a "Clear" function, but there's no sign that it has any ability to statically check that calling it is safe. That is exactly what we're talking about.
Nvidia: No public source code, and it's not obvious what actually exists here, beyond a compiler to Nvidia hardware.
Genode: This project is mostly written in C++. There is a very tiny, experimental amount of Ada in it (< 400 lines of code). There's certainly nothing that speaks to the questions discussed in this thread (safe dynamic allocation/deallocation, security, etc.).
ApexAda: This is a marketing brochure. Also this marketing brochure talks about "significant deployments in satellite communications, avionics, weapons control systems, and a myriad of other mil/aero applications," all of which - as discussed elsewhere in these comment - are applications that neither need nor want dynamic allocation (and are generally not exposed to deeply-structured untrusted input, for that matter).
Green Hills: This looks interesting - there's a POSIX filesystem and a UDP socket implementation, both of which (probably) want dynamic memory allocation. That is, at last, an application that needs dynamic allocation. Is it actually written in Ada? I see that the RTOS itself supports development in C, C++, and Ada, and they claim to offer a BSD socket implementation, so they at least have C bindings. I don't see any claim about what language it's written in, and Green Hills seems to do a lot of C work (see e.g. https://www.ghs.com/products/micro_velosity.html). If it's in Ada, then I think this gets the closest to what would be needed for the Linux kernel - do you know how their filesystem and socket implementations handle dynamic allocation? (I'll also note that they lost their certification many years ago, so if the answer is that they have safe code because they spent a whole lot of time perfecting a single version of an OS, then that's a very different problem space from Linux, which is constantly changing.)
Wind River: This is a marketing page without much detail, but it looks like it's a hypervisor to run general-purpose OSes on top of, not a general-purpose OS itself, which means it probably does not need dynamic memory allocation, or that if it does, the safety of deallocation can be verified by hand.
---
No Ada developer has done what's being done in this patchset, and it's pretty insulting to the folks putting in the work to claim it's been done and Gish-gallop a bunch of random links and expect people to put in the time to figure out what exactly you're claiming already exists. Everyone knows it's possible to write code in Ada. Nobody is asking if Ada can be used for completely unrelated purposes. You may as well link to GORILLAS.BAS and ask why the Linux kernel isn't using QBasic, a memory-safe language if you use the subset without PEEK and POKE.
Write - or point to - a kernel driver for a UNIX-like operating system written in Ada, that safely handles dynamic allocation. That's the clear bright-line goal. If you think that's not the right goal, say why.
>just that it's empirically likely to solve the problem.
Can you comment on what empirical evidence you based this statement on?
Yes, I can comment on this. The licensing is the same as any other submission to the Linux kernel. See https://www.kernel.org/doc/html/latest/process/license-rules... for Linux kernel licensing in general and also https://www.kernel.org/doc/html/latest/process/submitting-pa... for the meaning of the Signed-off-by lines.
> Can you also comment on how a third party can verify or audit the output of a compiler without a specification?
I imagine it's pretty similar to how you verify or audit the output of a compiler with a specification. If you can show me an example of the sort of verification or audit you envision, I can perhaps give you a more helpful comment.
We do not make any claims about verified or audited compiler outputs or compliance with specifications, and to the best of my knowledge, the Linux kernel does not currently verify or audit any compiler outputs, so I'm not sure I follow why I would be particularly qualified to comment on it, but I could maybe point you at some interesting research.
> Can you comment on what empirical evidence you based this statement on?
Yes, I did comment on this, in the paragraph you quoted. If you'd like me to expand on something can you clarify?
I must be missing the humor here. You can test against a specification to verify implementation. Testing an implementation against itself is meaningless. I would think that this is obvious. Also I don't typically use the word "empirical" to indicate purely qualitative assessments or intuition. You also provided a dead link.
There's no attempt at humor here - I'm just trying to get you to explain what, precisely, you mean by "test against a specification to verify implementation" so that we're in agreement on terms.
When I hear "test against a specification" and "verify implementation," I think about things like CompCert. But the Linux kernel does not use CompCert, it uses GCC or Clang. To the best of my knowledge, neither is tested against a specification and neither has a verified implementation.
Actually, when we first started discussing mainlining this, we had a bit of discussion about whether it was safe to allow rustc-compiled Rust code to link against GCC-compled C code. The specific worry was potential compiler bugs, of which the kernel has historically run into many. We eventually decided the risk was low, but the kernel has gone through periods where it insisted you use the exact same compiler for the kernel and for modules because of bugs. Those bugs have gone away through the normal human process of writing better code, not through formal verification. And when they existed, the kernel's response was not to reject those compilers, it was to work around the bugs.
Also, CompCert does not test against the C standard, which is written in English prose. It tests against the CompCert developers' attempt to re-express the C standard in terms of machine-checkable theorems. The C standard is confusing even to human readers, so it would be misleading to say that CompCert has verified their compiler against the standard as published by the standards committee. (Hence my claim that they're similar; you can verify a compiler of any language provided you write your own machine-checkable statements about what you expect it to do, and you still need other humans to read those theorems and convince themselves that they're the right ones.)
For those reasons, I'm trying to understand what your question is in a precise manner. Are you asking about things like CompCert? Because if so, I don't follow the relevance to our work, though again, I'm happy to point you at interesting research.
But Rust has a lot more going for it than safety. Maybe it shouldn't matter for kernel code, but the focus on developer "ergonomics," high-level functional constructs and pretty flexible pattern matching, supporting both ML-style and Algol-style syntax, very strong language-level tooling for building, testing, packaging, and generating documentation, plus the fact that the language itself is so well documented with great, accessible tutorials.
There's a reason it keeps winning Stack Overflow's most loved language award every year. It's easy to learn for people coming from almost any kind of programming background, supportive of newbs, and equally well-suited for application-level and system-level development, so it ends up with pretty widespread appeal.
So again, I don't know that any of this should matter for selecting what languages you're going to support in the kernel, but as a purely practical matter, it's a lot easier to ask contributors to use a language they already know and use elsewhere than it is to ask them to learn something new that they don't use anywhere else, which is what Ada and Idris and what not would be for most developers.
Note that some formal verification work has been done on the guarantees that the borrow checked provides (which led to finding and fixing an edge case bug), so there is a little more concrete evidence that just the "assurances of its advocates"
See http://plv.mpi-sws.org/rustbelt/ for more information, also the verification working group: https://rust-lang-nursery.github.io/wg-verification/
(Though, as pointed out by a sibling, there is a subset that is.)
You might want to look harder, because there is certainly a verified language subset (anything that can be written as LambdaRust, which covers a large and substantial swathe of the language).
If you're looking for a formally verified, optimizing compiler for any realistic language, particularly a low level language, you have pretty much one option: CompCert (I'm not sure where this large group of verified compilers you claim exists lives, but I'd love to see them). As someone who would love a verified Rust toolchain, I can't help but notice that very few safety-critical systems actually use CompCert in practice, which makes me a bit suspicious of claims that this is something Rust needs to be competitive in this arena.
Finally, I'll point out that verified compilers like CompCert only promise to translate correct C to correct machine code; they do not necessarily aid you in coming up with correct C in the first place. Miscompilations certainly happen, but not nearly as often as undefined behavior in the source program. So asking why people would use Rust when verified compilers exist for other languages is sort of missing the point.
Also, there is only undefined behavior in rust. The compilers behavior can change at any time for any reason any time somebody with commit rights thinks that it should. You cannot begin to assert provenance about some thing that can change at any time.
I somewhat doubt this, but even if it is true, the fact remains that people are not actually using it for the vast majority of mission-critical software. This suggests that the elimination of miscompilation bugs, which is what you get from formal verification of a compiler, is not high on their list of priorities.
> Also, there is only undefined behavior in rust. The compilers behavior can change at any time for any reason any time somebody with commit rights thinks that it should.
This is certainly not true of Rust in any practical sense.
I am not saying Rust doesn't need a certified version of its compiler that people can legally rely on for stuff like this (as opposed to practically). I am saying that this objection has nothing to do with the existence of a formally verified compiler, which is also not in practice used to compile mission-critical C programs. I think you are confusing certification of a compiler for mission-critical software, with formal verification of the language's compiler, semantics (including type safety of the safe subset--a proof that would not go through for C as it explicitly unsafe), and standard library.
It's not like Microsoft isn't pushing Rust on their own platform. Every time a Windows driver does a use-after-free somebody swears at Microsoft for causing a bluescreen.
At very least the Azure Sphere OS team should finally adopt Rust, instead of doing security marketing with the 7 layers of whatever, and then shipping a C only SDK.
It is like they are absent of the company security efforts, while trying to sell us a story of how much they care about security.
If I remember well, only 512KB is actually usable by OEM applications on that platform.
I was using C++ on MS-DOS.
Not even C++ is supported on the Azure Sphere OS SDK.
Ironically they usally tend to evade the question when asked about it, to the point they ended up writing a very long blog post trying to justify C, and it had nothing to do with space requirements.
Rather industry expectations and other marketing speak.
So instead of including a custom runtime they just bailed out...
Anyway just search for Jason Turner's talks on targeting Commodore 64 with C++.
No. They are only using C. They are not using anything else for "unknown" reasons.
You can write bloated Rust applications in pretty much the exact same ways you can write bloated C applications (generics, runtimes, macros, ...).
And you can write thin Rust applications in pretty much the same ways you can do so in any other language, including C.
I don't have enough experience with Ada to comment on whether its verbose and keyword-heavy syntax succeeds in this goal.
So that's like... your opinion, man.
SQL is another 'wordy' language. There are many, many things wrong with the SQL language, but I wouldn't say its wordiness is one of them.
For me, its always the var/function names and comments that determine how readable a program is.
Rust has opt-in memory safety issues but outside of those, does not require a GC.
Rust does not have Ada's constraint system. In that way, rust is easier (and arguably less safe) to program.
Ada didn't get the same love as Rust for the same reason D goes pretty unloved. The problem systems engineers needed solved was memory safety without a GC. Neither D nor Ada solve that problem.
AFAIK short of SPARK:
* Ada has pointer arithmetic, you can overflow buffers or read out of bounds.
* Dynamic allocation is straight unsafe and unchecked (unless you have a GC, in which case… you have a GC), you can use-after-free and double-free if you copy pointers beforehand.
* Accessing uninitialised variables is not forbidden.
* You can dereference null pointers (though it should cause an exception so I guess that's memory-safe).
Not true. Arrays have bounds and those are checked. However, you can do arithmetic with the type ptrdiff_t of the package Interfaces.C.Pointers. You can also do an Ada.Unchecked_Conversion on a pointer that you get from a C function. Obviously that's unsafe.
> * Dynamic allocation is straight unsafe and unchecked
If allocation on the heap fails, it will raise a Storage_Error, but you can catch it. Also, the language has a lot of restrictions on when you may copy and store a pointer.
> * Accessing uninitialised variables is not forbidden.
True, but there are pragmas like Normalize_Scalars (to initialize them to invalid values) or Initialize_Scalars.
> * You can dereference null pointers
True, but dereferencing will raise an exception. Also you can define non-null pointers like this:
type Foo_Ptr is not null access Foo;
procedure Bar (Value : not null Foo_Access);Static memory allocation is totally fine if the thing you're building is, like, a control system for a jumbo jet. You know exactly how many engines you have, how many wings you have, how many pilots you have, etc. Your autopilot has a fixed set of parameters: heading, speed, altitude, etc. You know exactly what tasks you're running: watch engine performance, watch the altimeter, watch cabin pressure, find the instrument landing system and line up with it, etc. If you have some new instrument to plug in, you get a new version of the flight control software, and you're certainly not plugging in new things mid-flight. Even if you aren't running all the tasks at once - e.g., you're usually not landing the plane - you want to guarantee you could turn on any of those tasks. You never want the possibility of running out of memory when you try to land, so you want to allocate memory for landing the plane at design time that's always reserved for landing the plane, and nobody would call that a "memory leak."
A general-purpose OS like Linux is different. You can create as many processes as you want. You can mount as many filesystems as you want, of various formats, whenever you want. You can plug in devices while the system is running. You can add drivers to a running kernel and remove them too. You can configure network connections, turn swap on and off, etc. Inbound network connections can show up at any time. So it needs dynamic allocation.
Ada is fantastic for problems like a flight computer. But it's a different sort of problem. For a general-purpose OS, you are allocating and freeing things in response to user input, and you want to make sure you neither leak memory nor use memory after it's been freed.
Rust is very good at solving that specific problem without the need for a GC. That's what people mean when they say "Rust doesn't use a GC."
Ada may be a good language, and GNAT may be a fine Free and Open Source Ada compiler, but Ada isn't a new programming language. Software developers only chase shiny new things. Ada is held back by its beginnings as a niche language with expensive compilers, it doesn't matter that this hasn't been the case for decades now.
That kind of thing happens; we're in a profession that is made up out of people.
I think one big stumbling block people had in adopting ADA (along with there not being a good open source toolchain for a long while as noted here) is that it just looks different to people that primarily use C and similar languages, and that puts some people off.
That's sad, and I would rather it not be true, but it's become increasingly obvious to me from how people treat numerous newer languages I've seen pop up over the years.
I'd be more than willing to give Ada a chance, but when I looked at code samples I definitely had to fight an automatic response of thinking it was something out of date based on how it looks.
It's not that there aren't successful languages and have a different look and feel, it's that there's pushback against them when they are targeted towards the niches C and C++ are already strong in. Obviously many languages exist and many have become popular, but certain niches are more resistant than others and sometimes it's for non-rational reasons.
> I'd be more than willing to give Ada a chance, but when I looked at code samples I definitely had to fight an automatic response of thinking it was something out of date based on how it looks.
That's part, but not all, of what I'm talking about. The style C follows is damn old itself, sometimes predating the languages you're thinking of as out of date.
Maybe also, but not only. In contrast to Rust, Ada docs have been awful, and as long as you didn't get a commercial toolchain you had to play with some gimped version (and the really free version didn't come out until the language was 20+ years old), the build system was as “good” as C, people complain that Rust is verbose, but Ada was still a notch over, the memory model of Ada is a mere reflect of Rust's one, and (maybe I'm being subjective here) Rust's stdlib and philosophy is leagues over Ada's (Option, Iterator, closures, Result, ...).
Ada had the potential to be a Rust 30 years before, but it was never the goal of its owners.
“I hope to see a switch from C to VB very soon. I've already spoken with various luminaries in the C coding world and most are eager to begin to transition. Having just gotten off the phone with Mr. Alan Cox, I can say that he is quite thrilled with the speed increases that will occur when the Linux kernel is completely rewritten in Visual Basic. Richard Stallman plans to support this, and hopes that the great Swede himself, Linux Torvaldis, won't object to renaming Linux to VB/Linux. Although not a C coder himself, I'm told that Slashdot's very own Admiral Taco will support this on his web site. Finally, Dennis Ritchie is excited about the switch!”
I have distinct feeling this would benefit from waiting a little bit more to have various kinks straightened out.
> Like it was mentioned in the RFC, the Rust support is still to be considered experimental. However, as noted back in April, support is good enough that kernel developers can start working on the Rust abstractions for subsystems and write drivers and other modules.
From my point of view what counts is actual kernel development. If you are going to develop kernel because it uses cool new language but you would not if it was still C, you are probably not the right person.
Now, I think Rust has potential to be good kernel language, I am mostly concerned with toolchain, ability for people to read the code with comprehension, ensure correct code, etc.
1. rust is easier to read than c/c++
2. The toolchain is llvm based and much nicer to use than c/c++. Hell your build scripts are also rust
3. Given the choice to develop in c or rust I would choose rust
4. In my own experience I have found rust to be simpler to Deal with correctness than c/c++
(I have been both a rust and c++ developer)
Rust may or may not be easier to read than C++, depends on your experience.
But it is not true it is easier to read than C. C is objectively much smaller, simpler language than Rust.
> 4. In my own experience I have found rust to be simpler to Deal with correctness than c/c++
That is the whole point of Rust. And also to do this with minimum runtime overhead.
Brainfuck is an even smaller, simpler language, it has only 4 characters!
Looking at the following function definition, can you figure out how to use it safely?
int foo(char* text, char* out,
char* s, char* x);
fn foo(
text: String,
out &mut String,
s: &str,
x: &mut str,
) -> i32;
Having a more "complex" way to represent in the typesystem something that C only bothers to call "a pointer" makes things better: safer, easier to skim, explore and understand an API, and allows the compiler to catch API misuses.
- GUI code
- Generic datastructures crates coupled with lifetimes, macros and attributes
Why's that? C is a tedious language that makes you do so much manual bookkeeping, I don't think I'd ever write it again (if I needed C for some reason I'd write a program to generate it for me rather than doing it myself). I think any decent developer would be put off by C to some extent. Sure, the people who really want to make a kernel may put up with it, but an arbitrary enthusiasm barrier is not the way to attract good contributors.
C is tedious but kernel programming isn't about spewing vast amount of code anyway. It is more important to be able to read it, understand it and ascertain it is correct. This is one of main reasons for why C++ is not there.
Except for, you know, all that Undefined Behavior stuff, which makes it practically impossible to write correct programs even for very skilled developers, or the lack of abstractions, which makes it hard to safely re-use advanced data structures.
"C is a simplistic language" would be more accurate.
Code is either clear and simple, or extremely obscure and safe, or kinda in between is complicated but not complicated enough such that its not safe.
"Simplicate and add lightness" aerospace mentality people like C.
At some point any project expected to do ever more complicated tasks, like a kernel, will be unable to use simple C and will have to take the hit of a more complicated language.
Simple is good when matched to simple tasks; my microwave oven or TV should ideally never data network with the external world and never malloc memory and never do string processing. C makes an amazing toaster or car automatic transmission controller.
Which one would you rather defend yourself with?
Call me indecent. I use C for firmware development running on small microcontrollers and consider it perfect for this particular task. Not put off by C for this task at all.
Any decent developer would be put off by any language to some extent.
Personally, I'm more put off by Rust than C (they put me off for different reasons). Currently, the one puts me off the least is Zig, but it still has many details that I don't like.
Also, simply by having god damn namespaces more readable names could be used at no downside.
You are taking this the wrong way around. It's not about people contributing to $PROJECT because it uses $COOL_TECH, it's about people being turned off from contributing to $PROJECT because of $MEH_TECH.
If Linux was written entirely in assembly, we'd be having the same argument about the introduction of C.
Also BCPL, the basis for B intepreter, was originally designed to Bootstrap CPL, not really as a daily use language.
https://github.com/rust-lang/rust
The git repo has over 150,000 commits, 3,000 contributors, and 9 years of release history. The last 6 years of which have been post 1.0.
Also, Mozilla has been writing large parts of firefox in Rust since ~2017. There's some interesting writeups on hacks.mozilla, including this one on reducing CVEs with Rust
https://hacks.mozilla.org/2019/02/rewriting-a-browser-compon... https://research.mozilla.org/rust/
I do embedded development on ARM Cortex-M3 and while it is possible to do this in Rust, I have experienced a bunch of issues with the toolchain and I have decided to stay on C until toolchain improves.
Don't get me wrong, I like Rust. Just don't allow liking something get in the way of practicality.
Microsoft actually did do that in the 90's.
Feels a bit like the "new more efficient battery technology" recurring post.
edit: to precise it's not that I'm not supportive of the effort, just that it would be nice to just have a post when it's actually merged. The title is a bit deceptive
The Linux kernel has become one of the most, if not the most important piece of IT infrastructure. Progress of any further developments of it as well as the security itself are prime concerns for all users. So far the development was solely limited to C, with all the tradeoffs. It is highly interesting to see whether a more modern language can be incorporated and the Rust effort looks very promising. Both from the language capabilities as well as the progress being made.
Ada, C++, Java, Objective-C, Swift and yes Rust.
Note that the kernel still uses a lot of GCC specific syntax; it's just that LLVM/clang was laboriously made to understand all of that (with a few exceptions where the kernel was changed instead). And when they were almost done, the kernel started requiring another new GCC specific syntax, so they had to chase after that too.
In the long term, LLVM/Rust support for more architectures improves, and support for older architectures is dropped, this will become a larger fraction of kernel modules, but it will be a very long time until it is considered for something like ext5.
[1] https://github.com/alkhimey/Ada_Kernel_Module_Framework/blob...
The point is to integrate the Rust language to the Linux infrastructure so you can merge rust drivers to the official code base.
They've been accepted into linux-next, which is a staging ground and used for testing, but that by no means guarantees that they will ever be accepted into the mainline kernel.
It seems to me like a Trojan horse from G and MS that pursue their own interests with their own OSes.
I don't know much about Rust so can't comment on that part. But agreed on the Scala part. Having seen Apache Kafka use Scala in the early days, when Scala was considered "hip", alongside Java code in the same codebase, I can relate to the mess it created and continues to be even today. At some point there was a plan to move away from Scala in Kafka, not sure where that reached, I stopped following that project since a few years now.
Given all the CVEs that arise due to buffer overflows and UAF, it's not working very well. Safe Rust makes those classes of problems Just Go Away.
> Do you think inexperienced programmers could build it given a "safer instrument"?
Inexperienced programmers can get started without the near vertical learning curve C presents. Just wrapping your head around pointers scares many programmers off, and even once you grok pointers, attempting to use them correctly is a minefield. This makes kernel hacking daunting for the average dev. Rust gives them an onramp.
> How many junior devs understand intricacies of basic OS-level locking, not mentioning Paxos and other scary algorithms whose formal correctness can't be even proven in theory?
Not very many... but junior devs don't need to work on that stuff. They can make more peripheral contributions -- fixes to device or file system drivers, etc. -- while they're building their skills, completely confident that they will not make a class of stupid but very common and dangerous memory errors. Not so with C! As I said, not even great programmers can be relied upon to write safe C code -- the model presented by C is just that abstruse.
Just the opposite. We're going to see C deprecated for new code and efforts to move the existing C codebase to Rust. C is pretty much unsafe at any speed; Rust allows even junior front end developers to fearlessly contribute bare-metal code once they've learned the language.
Adding a 3rd language would be madness, Rust is already complex enough - so that window of opportunity will be closed.
Furthermore, for a language to be supported, it must have enough infrastructure (tooling, people, convention...) to support that language first, so people to review patches shouldn't be a problem.
Obviously, language should not be added willy-nilly, it must prove that it can solve some problem that the current set of languages can not solve efficiently, and base on that merit, it will choose to write/rewrite the modules that have the most of those problems first.
TL,DR: a decentralized codebase.
...
Also, why is this in big-endian order?
Let's just try to kill big-endian data, it's disgusting
and should just die already.
BE is practically dead anyway, we shouldn't add new cases. Networking
has legacy reasons from the bad old days when byte order wars were
still a thing, but those days are gone.
I'm half convinced that he was racing to be first to give it a seal of partial-approval to keep comments more on track.
To me, big endian makes a lot more sense - integers are stored in the order which you read them (mentally).
I do remember reading about how little endian enabled some type of optimization inside the CPU, but I forget the specifics.
Obviously this example is very contrived. This sort of thing was much more of a concern on 8-bit computers. But little endian still seems more natural to me.
Some of the other replies have minor technical reasons, but I've always preferred big endian for the readability. Having said that, I'm happy to part with the idea of big endian if it means an end to having 2 options to worry about. One thing that bothers me a lot about RISC-V is that the standard claims to allow either big or little endian implementations. Little has won and nothing new should support big endian IMHO. The benefits of either are largely irrelevant, but the existence of both is a problem. Or maybe it's that the existence of code that cares is the real problem ;-)
Specifically, the problem is with "carries". When you're adding (or subtracting, or multiplying, or dividing, I'll just discuss adding) two binary values you might have to carry a 1 to the next place.
If you've got a BE value and an adder stage smaller than that value (say, a 32-bit number and 1-bit adder stages) you have to carry a 1 many times to output the result. If you're receiving the value in BE order 1 bit at a time you can't start the computation until you have the LSBs of both values, since if they're both 1 they'll affect the second bit by their carry. So you're stuck waiting for the entire value to start the computation. Further, during the computation you have to wait for every 1-bit adder in sequence.
There are "fast" adder designs that don't have to wait for every bit, but can instead work on groups of multiple bits with a carry-out at the end of the group. So if you've got an 8-bit group size, you'd have at most 3 carry delays during the computation of the output. For BE, you'd have to wait for all 4 bytes to be received, then wait for all the carry delays. For LE, you can start the computation as soon as the first byte is received, saving some time.
The larger the adder group size the more die area is needed, the stronger the drive strength of the transistors needs to be (bigger fan-out), and the slower the maximum clock of the overall system. On the other hand the bigger the group size the fewer carry delays, so addition can take fewer cycles. Most CPUs and MCUs implement single-cycle addition of their word size. Some CPUs even implement single-cycle multiplication at their word size.
There are also non-Linux cases, mostly microcontrollers. EG the Arm Cortex M0 doesn't have a hardware multiplier, the M0+ does.
And then there's that one guy who got Linux running on an 8-bit AVR by emulating a 32-bit ARM and running it on that[1]. I'd consider this a silly edge case. Too fun not to mention though.
[1] https://dmitry.gr/?r=05.Projects&proj=07.%20Linux%20on%208bi...
Yeah, this isn't true. Low level hardware receiving data still likes to use shift registers:
* Zero shift reg
* clock in one byte, shift into shift register
* Clock in next byte, shift into shift register shifting the previous byte left one
* Repeat for as many bytes as you have
Processors can't agree on endinaness, but network protocols have. "Network byte order" is a standard thing that is almost completely universal across communication protocols.
What he is saying is that for pretty much everything else (read: typical CPUs) it's all little endian now.
Basically, I understand that shifts for incoming serial bits is very convenient. But as long as you specify the "word size" used in your local memory, I do not see the problem: just use 8 (for instance) shifts for your 8-bit word and then go to the next memory location. Why is this fantasy wrong?
It is the default byte order in Java bytecode.
Dangling pointers are not a thing.
No double free.
Forces you to think up front about what are the possible state something could be in, and so you won’t get runtime or weird UB.
lastly, to be more pedantic for everyone who is at Linus-levels, it’s like all the best static analysis and warnings/errors all rolled up natively into the language. Please don’t bombard me with “but I can just run clang’s X and get the same thing”. No you can’t, and if you did this forces you to run it regardless.
Yes, we can.
Then you still have to run extensive fuzzing and testing to trigger different paths and whatnot.
Or you could just use Rust.
The ownership/lifetime system is basically novel to Rust (well, it's based on Cyclone, which would probably have been an interesting choice for the kernel too because it's based on C, but Cyclone is no longer actively developed), and it allows you to get the performance of manual memory management (no refcounting required) without either leaking memory or hitting use-after-free bugs.
Rust's ownership system also tracks shared/immutable references and unique/mutable references at compile time, avoiding a whole bunch of setups for creating race conditions or otherwise corrupting data.
> Rust's ownership system also tracks shared/immutable references
But shared/immutable vs. unique/mutable references aren't the same thing, they're something more interesting. Rust has three types of ways you might be able to get at a variable: you could own the variable, you could have a shared/immutable reference to it, or you could have a unique/mutable reference to it. The Nim proposal only distinguishes the ownership case (responsible from freeing the data when it's done - basically RAII or __attribute__((cleanup)) or autorelease) from the unowned case.
Having shared/immutable references gives you a neat feature for achieving correctness, not just memory safety (and I strongly disagree that the idea of "memory safety" in that post really counts, incidentally, because by that definition Heartbleed wasn't a memory-safety bug, but for the sake of argument let's say it does). The compiler prevents you from having more than one mutable reference at a time. It also prevents you from writing data you own while someone holds a reference to it, or even reading data while someone holds a mutable reference. So if you're writing code where you pass a reference to some other routine, and that function ends up holding onto it for future use, you won't be able to modify it. That prevents logical bugs where code doesn't realize that data has changed or conversely doesn't realize that a change hasn't taken effect.
Again, this is not solely about memory safety. This is about easily writing correct code, of which memory safety is one aspect.
You can think of this as basically type-level reader/writer locks. At any given time you can have multiple readers and no writer, or exactly one writer. While runtime memory safety is one thing that reader/writer locks can do (e.g., preventing you from calling realloc() while other people still have a pointer that they intend to read), they're used for a lot more than that.
Furthermore, Rust's approach requires no runtime overhead and preserves correct behavior. Nim's approach requires refcounting, and the post argues that refcounting could be disabled in a release build if you're confident your code is correct.
Nim had the concept of borrowing from very early on. It's not a bolt-on, it's part of the design philosophy. But it's had consistency issues. The move semantics in newer Nim improve on it.
I would say one of the major things Rust offers over Nim at the moment is the implementation quality of the Rust compiler, standard library and type systems is generally of higher quality and more consistent. Things are thought-through more completely in Rust's design, and tested more against real world problems.
The multi-threading in Rust is also of much higher quality. It's just buggy in Nim.
Nim however has a much cleaner syntax, and the quality issues could change if the community really wanted that (I don't think it does).
They both have slow compile times for large programs.
So I'll instead just give reasons why people might prefer to program in Rust:
- Memory management / safety (no GC, variable lifetimes are tracked by the compiler, no dangling, no null pointer ...)
- Ergonomics (built-in unit tests, feature-full std lib, unicode strings, modern dependency manager, more formalized macros ...)
- Functional-lite (you must explicitly discard unused enum variants, the "null" value is an enum variant ...)
Long story short: Rust makes it really easy to write sound software. I think you see that on crates.io where Rust packages are hosted. Lots of packages quickly reach maintenance mode and receive very few updates, but remain extremely useful. Once you've written (compiled) something in Rust, you feel much more confident that is is free of latent runtime bugs.
Without commenting on whether or not this is necessary or good for the Linux kernel, I'll say that there are probably many devs out there that feel like working other non GC languages feels like pulling teeth after having worked with Rust. And that might be part of the pressure.
Though dealing with rust dependencies by hand is usually nicer than dealing with C dependencies by hand because the build process is more standardized.
But there are some significant improvements WRT memory management/indirection.
Type inference
Language support for tagged unions (which they call enums for some reason), which among other things results in error handling which is "out of channel" instead of magic return values.
Speaking out error handling - syntax for easily forwarding errors to the calling function.
Generics, and therefore
- Libraries of collections with a rich collection of methods implemented for you.
- Library support for lazy iteration
- ...
Method (postfix call) syntax allowing for readable chaining
Hygenic macros
Compiler supported test harnesses
Compiler supported linting (that is really good)
Compiler supported automatic formatting (that is really good)
Compiler supported vtables
(pointer, length) strings
Proper namespaces
And so on and so forth
Probably because the simplest case of Rust enums (without any data other than the tag itself) can act similar to a C enum, to the point that you can "#[repr(C)]" it and use as FFI for the equivalent C enum. One understated advantage of Rust for C programmers (which is on topic given that the Linux kernel is written mostly in C) is that its syntax and keywords have many similarities with C, starting with the use of braces for blocks and semicolons to end statements.
> and use as FFI for the equivalent C enum.
Only if you're willing to accept undefined behavior if the foreign code passes you an unknown variant. For actual ffi work I've ended up using a macro that makes a "#[repr(C)] struct EnumName(u32)" with a bunch of constants for the values.
If there are any values that don't fit in a u32, I would need to use a different int type. C enums automatically use bigger int types if they don't fit in the smaller ones.
If I was on an architecture where int was smaller than 32 bits it would also be incorrect (except by coincidence that 32 bits was the smallest type that could hold all the values). I could "solve" that by using libc::c_int instead, but that only sort of solves the issue because if libc::c_int is 8 bits, and the enum has a value 257, it's just not correct anymore.
I suppose it's possible that some abi treats a struct containing an int differently from an int, I doubt it though. If that's the case, #[repr(transparent)] instead of #[repr(C)] would be slightly more correct.
In some ABIs this is not true, a C enum is a char if the constants fit. Your code that assumes enums are stored and passed around as 32-bit integers is not entirely portable.
Ultimately, at least at the time, there was no other option that had better compatibility. Now I suppose you could use MaybeUninit<#[repr(C)] enum> or something. Realistically I don't have much confidence in rustc properly matching enum sizes with C implementations that don't follow the typical behaviour here, so I doubt that would be worth it.
I guess the linux kernel is generic enough that it might end up on some such platforms, I guess in that context I'd just bite the bullet and accept undefined behavior if passed an unexpected enum variant, it's not like that should happen in a coherent system like the linux kernel (vs my use case which was wrapping some library that definitely might chose to add enum variants).
Isn't that what `#[non_exhaustive]` is for?
For example your Rust function that always returns 42 should probably be:
const fn fortytwo() -> i32 {
42
}
const fn fortytwo() -> i32 {
return 42;
}
2. Ergonomic use of functional idioms like sum types (Rust's enums)
3. Memory safety without GC overhead - Windows, Linux, curl, over and over again 40% of vulnerabilities in larger C projects are found to be preventable if there was a memory safe language
Also, if you're "just starting" don't burden yourself with legacy stuff. Unlearning that when starting with Rust isn't a cake walk.
Rust is a pretty good language but it's not the most used. Also you have to take into account that you can't use Rust on every field.
If you're looking to be an Android/iOS/web dev, Rust is not the best option.
Learning a language people do systems programming in can be a good experience but it doesn't need to be Rust for you to get that experience.
If you want to learn Rust because you want to, that's fair too.
You may edit an HTML template here and there.
Maybe even some configuration work for CI/CD.
Rinse, repeat, with whatever new language/framework is hot right now.
Basically, you will be solving the same set of problems on every job, but in more convoluted and unnecessary ways, because the guy who worked at Google says that's how they do it over at the FAANG.
And I don’t even mean it’s a tragedy : I largely prefer doing CRUD-for-rent in a great company than technically fulfilling job under asshole management in some unstable startup.
Now, if you plan to do any kind of systems programming, C and C++ are unavoiable, as that is what the majority of the jobs still expect one to use.
Web technologies? CI/CD, IT automation, etc.?
Or something more like operating systems, robotics, hardware, drivers?
Or do you want to self-educate for the sake of it, to learn different paradigms and be a better programmer in general? (Everyone wants to do this, but not everyone has the time).
Golang and Python are incredibly popular in the automation/"glue code" space.
Java is popular in enterprise development (think banks).
Systems languages are popular if you're developing... systems.
For fun and interest? Why not?
https://blog.rust-lang.org/2021/05/10/Rust-1.52.1.html
"The Rust team has prepared a new release, 1.52.1, working around a bug in incremental compilation which was made into a compiler error in 1.52.0. We recommend all Rust users, including those currently using stable versions prior to 1.52.0, upgrade to 1.52.1 or disable incremental compilation. Guidance on how to do so is available below."
Is this mature enough to be bring in the kernel ????
rust is more of a secure c++, that has its own object oriented pattern, which linux kernel uses a lot but in C.
pronouns are an alternative
Using pronouns can very quickly become confusing. When someone says "it" are they talking about rust, linux, the kernel, linus's temper? Using "rust" instead of "it" completely removes that ambiguity.
Rust attempts to solve the worst of pointer problems - I think overusage of pronouns would undermine the whole Rust effort.
here's the word count
rust: 13 in: 11 the: 10 to: 8 a: 6 and: 5 for: 5 linux: 5
On a more serious note, I am curious to see what license RIIR Linux will have.
I'd rather see a new kernel optimised and written in Rust from scratch rather than dragging the legacy skeletons around and also bringing the vulnerabilities with us.
That'll take a momentous amount of effort and it'll take 20 years to get close to the adoption Linux has today. I'd rather see if we can improve Linux in that timespan.
> On compilers, we would like to mention all the work that has been going in GCC Rust (a GCC frontend for the Rust language) and `rustc_codegen_gcc` (a `rustc` backend for GCC). The latter now passes all `core` tests, and the former is now working on traits. We continue to track their progress as they may become the best way to have GCC-built kernels with Rust support enabled.
Writing a new kernel with all the capabilities of Linux is an absolutely enormous undertaking, billions of dollars worth: "it would cost approximately $14,725,449,000 dollars [...] to rewrite the existing code." -- https://en.wikipedia.org/wiki/Linux_kernel#Estimated_cost_to...
let's outsource it, so we can cut the cost by half, at least
I just saved 7 bilion dollars and probably need to become manager
30 years later, there'll be a kid in university somewhere about to send a thoughtmail that says "I've written a version of..."
There is Redox, which was intended to be practical, but I'm not aware of much public interest in that. Phillip Oppenheimer blogged about one too, I think, but that's explicitly academic.
In my experience with mid size projects the key to success has always been controlling complexity. I can only imagine that this is even more true for large projects.
Generally the complexity an additional language brings to the table is immense in several aspects.
Specifically Rust has it's own problems and drawbacks.
Looking back at the history of Rust and Firefox/Servo, introducing Rust was overall quite a disservice.
Now to contribute not only you have to learn C, but also Rust and get their complex dev environment setup..
This fragmentation is very bad, seems like they embrace the linux distro spirit.. what's gonna be tomorrow? zig support?
Lot of effort wasted in making things more complicated, definitely not a positive move
"The linux-next tree is the holding area for patches aimed at the next kernel merge window. If you're doing bleeding edge kernel development, you may want to work from that tree rather than Linus Torvalds' mainline tree. "
Being able to do kernel development in Rust should lower the entry barier into kernel a lot. Because it has more high level constructs and especially for all the safety Rust brings. Any competent programmer should be much more comfortable to start kernel dev with Rust than with C.
You now are going to have two languages + bolting on a kernel riddled with chronic security holes and still supporting dead architectures and 32 bit systems.
Well see if brown-fielding Rust into the Linux Kernel is all worth it for them.
Rust feels like a modern language, like Go or C#. C is barely better than FORTRAN.
I'm massively more productive in Rust and I don't have to worry about a million ways of causing UB or memory bugs.
Everything from dependency management to testing to cross compiling are dreamy in Rust compared to C.
Dependency management, packaging, and testing are what really made Rust take off. The best of any language I've used.
C++ support could have been viable a decade ago had it not been for strong egos on LKML refusing to admit they were wrong. Rust support in Linux is at least partially a face-saving way of backtracking on C++ in the kernel.
Rust is great, but with the sole exception of Rust's reified lifetime management, C++ is the better language.
Why? How, precisely, does C++ hurt?
> It has certainly been tried and in the cases I know of, regretted.
Did it? Fuchsia didn't regret C++ in the kernel. GDB didn't regret moving to C++. (I know GDB isn't the kernel, but it's systems-y.) Android's libc, Bionic, is also written in C++.
Every single argument I've seen against C++ in systems code has been specious: people act like typing "g++" instantly implants a copy of the GoF book in your head and turns you into a design patterns robot. "I don't like it" is not a technical argument. Neither is a profanity-laced rant, as satisfying as it might be to pound out such a thing.
You can just use C++ as a safer C. Linux would have gotten huge rewards in both safety and developer efficiency if LKML hadn't spent a decade doubling down on their nonsense arguments against C++.
C++ is fine for systems code done tastefully. GDB, GCC, libc, java, go nuts! Kernel, not so much.
There's a few things going on. You have to agree on a subset of C++ which is a royal bloody pain in the backside at best. It's like a bracing style meeting where the issues actually matter. Some of this is decided for you, you can't have exceptions. Now you can't have operator overloading either. Now constructors are a bit of an issue to consider fairly deeply, "YOU'RE KIDDING YOU'RE SO WRONG I HATE YOU!" And I don't even know what you think about it yet! ;-) Now do that with thousands of contributors. Nasty. Google and Fuchsia? I think they've already got rid of most contributors with the CLA. https://cla.developers.google.com/about/google-individual I couldn't find an actual license and didn't feel like curling to bash to find out what's in the code. It's not surprise that most who try it have a microkernel based system. It is doable. It's been done! Criticising Linus for refusing to go that way seems a little naieve to me tbh. He can point at considerable success as a result of his fundamental approach. That may not be the end of it. But it clearly wasn't a disaster. Critics haven't managed to achieve the success the early days of linux yet as far as I'm aware.
C++ compilers have universally sucked incredibly hard until fairly recently when gcc and clang really lifted the standard. No the crooks at intel and the way they sabotaged their customers to make their competitors look bad do not count. They lost the right to be taken seriously at that point.
C++ for cross platform has been horrible until very recently. A = B + C + D; You know exactly what that does in C without thinking. C++ not so much. Having those extra layers of thought required between the reader and the machine is unpleasant for OS stuff.
C++ has its place. Many of its features are extremely useful and seductive. Many can be used tastefully in userspace to make better, stronger, more reliable code. The Kernel? I don't see it . You want to do dynamic dispatch in C. You can see exactly how that will work on the machine. Virtual functions. No, there's magic there. The implementation isn't guaranteed and can change.
There are simply more corner cases, gotchas, foot guns and nastiness where you really don't want it. Kernels are hard enough.
Well put. Given the style of C used in the linux kernel, it should have been possible to restrict the use of C++ in the same way. Yes there is lots of bad C++ in there, but it doesn't have to make its way into the kernel. The maintainer s can reject code that is uses inheritance, or templates in the wrong way, etc.
It's like believing you shouldn't use C in the kernel because it's possible to write lines that are over 80 characters long.
And no, it wasn't C++ that killed Symbian or BeOS.
Windows NT kernel was all C apart from bits of the graphics layer - friction from that team's use of C++ is a theme in /Showstopper/.
Naturally they did not rewrite everything from scratch, just opened the door for new code to use C++.
As for BeOS, well I never seen the kernel and Haiku doesn't count.
It was quite difficult to write, and new hires had to essentially be retrained. Expecting an app ecosystem to be built on Symbian C++ was unrealistic. It's the lack of an ecosystem that killed it really.
C++ haters seem to always try to use the move that unless one isn't using 100% of language features and standard library, one isn't using the language.
Yet the same measure doesn't apply to C, when it is crippled to use in kernel code.
For Rust to be judged differently is a huge signal, I think.
ARM "operating system" is hardly used extensively.
Arduino's are embedded devices and can be programmed in any language but are hardly applicable across the vast range and scale of Linux deployments.
IBM's primary operating systems that aren't Unix based are written in Assembler or even RPG.
IBM mainframes are written in mix of PL.8 and PL/S originally, with newer code being C++.
ARM apparently doesn't have any issues getting their stuff used in medical devices.
God I want to see the source code of an operating system written in RPG now.
I don't know about "better" in the general sense, but for the specific use case of the Linux kernel, Linus has identified some entirely reasonable objections (though aggressively worded) to C++: https://lkml.org/lkml/2004/1/20/20
- the whole C++ exception handling thing is fundamentally broken. It's
_especially_ broken for kernels.
- any compiler or language that likes to hide things like memory
allocations behind your back just isn't a good choice for a kernel.
- you can write object-oriented code (useful for filesystems etc) in C,
_without_ the crap that is C++.
There are good, principled reasons to accept Rust and continue to reject C++.
(There are also good reasons to believe that C++ doesn't solve any of the problems with human inability to write correct C, meaning that it wouldn't be worth the cost of adding a new language: https://alexgaynor.net/2019/apr/21/modern-c++-wont-save-us/)
> the whole C++ exception handling thing is fundamentally broken
Not an argument. It's the closest thing in Linus's excretion of unfounded technical opinion to an argument, but it's nevertheless not an actual case for anything. Don't like exceptions in C++? Don't use them. (That said, exceptions aren't "fundamentally broken" no matter how many times Linus repeats that baseless slander.)
> any compiler or language that likes to hide things like memory allocations behind your back just isn't a good choice for a kernel
This is a general-purpose tirade against any abstraction whatsoever. Good software engineering is all about judiciously "hid[ing] things", including memory allocation. And as it turns out, Linux has plenty of abstractions, some of which involve memory handling. Also, you don't have to use any particular abstraction with C++: you get to choose the abstractions. The language doesn't force any on you.
Besides: Rust can do the same sort of abstractions that Linus claims disqualifies C++ as a "good choice for a kernel". It's almost as if C++ was fine even back in 2004 and that the only difference between then and now is that Linus has matured.
> you can write object-oriented code (useful for filesystems etc) in C, _without_ the crap that is C++.
This is a restatement of Linus's opinion, not an argument. You can write everything in assembler too: why take on the baggage of the heap of crap called C?
I think there exists a solid and reasonable argument along these lines, even if Linus did not phrase it, though, which is what I'm getting at. (And I suspect Linus and others do believe that argument, even if they're not good at communicating it, so it would be worth engaging it.)
It's true that you could write C++ without exception handling, and that you get to choose the abstractions. But it matters what the language makes easy for you - what is idiomatic and conventional in the language - what programmers who use that language have come to expect their libraries to do.
Otherwise you could just say that we can solve our string problems in C without switching languages by simply using length-prefixed strings and buffers and not relying on NUL termination. After all, the kernel doesn't use a userspace standard library, and even in userspace there are string-handling libraries in C that avoid those problems (e.g., http://bstring.sourceforge.net/ or https://github.com/antirez/sds).
But the fact remains that the kernel has its own implementations of strcat and sprintf nonetheless (in the lib/ directory), simply because C programmers expect them. And people call NUL-terminated strings "C strings" and length-prefixed strings "Pascal strings," even though you can do length-prefixed strings just in C and NUL-terminated strings just fine in Pascal.
It is very hard to write abstractions in C++ that work the way the abstractions in Rust work and are still natural and easy to use. No one is disputing they're possible to write. But if you look at the blog post I linked, it demonstrates that even with the benefit of decades of experience in C++ and knowledge of what Rust is doing, and even with additional language features, and even with the input of people who did in fact choose their abstractions (std::string_view was based on libraries local to Google, LLVM, and Bloomberg), you still do not get the level of safety that the equivalent Rust feature has.
Concretely,
1. C++ has language-level features (most importantly, try/catch/finally) that make error handling via exceptions easy and natural to use. Rust has language-level features (most importantly, pattern matching on sum types) that make error handling via a result type easy and natural to use. You can of course choose not to use exceptions, but you're giving up a good language feature and you don't have language support for the thing you want to do instead. This is much like saying "Don't like NUL-terminated strings? Don't use them" - technically true, but have fun avoiding string literals.
2. This is not a general-purpose tirade against abstraction. It is a very targeted tirade against implicit conversions to heap-allocated types, which are natural and idiomatic in C++. See my comment https://news.ycombinator.com/item?id=27749301 elsewhere in this thread for details on exactly what Linus is objecting to and why it is entirely reasonable for a kernel to object to it. Again, you could certainly decide to avoid implicit allocations in your nonstandard library, but it is not idiomatic C++.
3. C++'s object model is legitimately quite complex, involving things like multiple inheritance, partially-constructed objects during the constructor, and phrases like "morally virtual base". It works - there's definitely a lot of working C++ in the world - but that level of complexity isn't needed by the kernel, and (certainly in my personal experience) it's easy to get into deeply confusing situations. The kernel also cares about things like fitting objects into cachelines. The kernel doesn't need multiple inheritance with the ability to have different offsets into an object depending on what pointer type you're using, but it does need the ability to specify that objects implement multiple interfaces that carry no data (what the kernel calls "operations structures," and what Rust calls "traits"). In C++, the only mechanism the language gives you for this is multiple inheritance. There are use cases where full multiple inheritance is useful, and C++ is preferable to Rust for such cases. The Linux kernel is not one of them.
It's true that you can do all the things Rust can do in C++ if you try hard enough. But I'd reiterate the point which you made clearly, that human beings simply do not seem able to write flawless C in practice, even though it is theoretically possible, because it is simply not idiomatic. From the blog post I linked, it seems like there's no reason to expect that asking humans to write flawless and unidiomatic C++ will work out any better.