Is Rust truly the first good candidate for Linux's second language? Why didn't older languages get this far?
It is the "first language" that Linus didn't object to. He hated both C++ and Ada. I am not entirely sure if he really likes rust, sees borrow checker as something important worth trying, or lost to the pressure from RESF ( Rust Evangelism Strike Force ) within the linux kernel dev community.
He is a proponent of C because he can imagine, or envision the generated assembly from C compiler. Something like a High Level of assembly. And something along the line of people who can write good C code tends to have all the scar tissue to prove it.
>What do you think of the projects currently underway to develop OS kernels in languages like Rust (touted for having built-in safeties that C does not)?
>That's not a new phenomenon at all. We've had the system people who used Modula-2 or Ada, and I have to say Rust looks a lot better than either of those two disasters.
>I'm not convinced about Rust for an OS kernel (there's a lot more to system programming than the kernel, though), but at the same time there is no question that C has a lot of limitations.
And I asked him once I think in a private email a few years back and he manage to gave me a long detailed reply that was like an essay ( Still very very grateful for his time ). May be he doesn't hate Ada as he did to C++, but it certainly feels he doesn't like it.
I would be interested to see his opinion on Zig, which I think really fits his liking. But may be after 1.0, which is still some way off.
The only major one I know of was some push for C++ which Linus wasn't a fan of: http://harmful.cat-v.org/software/c++/linus
One important factor to consider there, and for which Linux is wholly unlike the vast majority of other projects, even system projects, is that the userbase is just massive, and includes a lot of the world's infrastructure. That results in a rather different cost/benefit calculus around abstraction/performance tradeoffs than the ones for which object-oriented and functional programming were designed.
For example, I think that it's probably good for all of us that there's not really a whole lot of dynamic dispatch happening in the Linux kernel. I would generally prefer that my syscalls keep pointer chasing to an absolute minimum. But I'm hard pressed to imagine how a C++ style that avoids vtables like the plague could really have that much to offer over straight C. And it would simultaneously put extra code review pressure on the kernel team, since they would have to carefully watch to make sure contributors aren't introducing such things.
Similar concerns apply to Rust, mind. But it's much easier for me to see how Rust without trait objects could still bring something of value to the table.
unique_ptr / shared_ptr alone would be a massive upgrade over straight C. Simply being able to enforce ownership would be huge. So would just having namespaces. Things like move semantics, static_assert, and constexprs would then be the cherry on top. There's also public/private fields, friend classes, etc... to keep people from accidentally poking things they shouldn't be.
C++ is so very much not just "C with vtables."
But C++ has a reputation, deserved or not, for being overly complex. And ultimately Linus chose the devil he knew over something that by all accounts would have been better. Nearly every other large C project has moved into C++, even from other staunch defenders of C like John Carmack eventually relented that C++ was better for large teams.
Rust gets to hit the reset button here with a fresh reputation (and some extra safety & cool tricks), and Linus' views may have changed over time.
Examples include exceptions as the one true error mechanism, and the use of implicit allocation.
Rust doesn't have the former, and Rust's core doesn't have the latter (but std does), one of the stipulations from Linus when this began was that Rust for Linux needed to ensure it didn't sneak in implicit allocation. Thus, while a beginner's Rust program might have text += "Some more stuff"; you won't see that in the kernel -- because it won't compile, the core::ops::AddAssign isn't implemented on these string types in the kernel because it's implicit allocation.
As to move, the move semantics in C++ are really awkward. They're a nice optimisation of course, which is why they're provided, but you would not deliberately do it that way in a language which had move semantics from the outset, as demonstrated by Rust. To make move work with existing C++ software it has to "hollow out" objects when they're moved from. Rust needn't do that, if you move a Foozle from var1 to var2 in C++, it must carefully hollow out the Foozle in var1, because in C++ var1 will eventually get destroyed, so its contents must be valid and safe to destroy. If you move the same Foozle from var1 to var2 in Rust, the Rust compiler knows you moved it, so, it's gone, var1 won't get destroyed and no further attention to it is needed.
-fno-exceptions is very widely used and non-throwing variants of basically everything exist as a result. That's not a reason to avoid C++ in the kernel as a result. Especially with how heavily Linux already relies on GCC C extensions, there's obviously no hangups on being "strictly standard compliant." Not have exceptions ever been C++'s "one true error" mechanism ever. For example see iostream: https://www.cplusplus.com/reference/ios/ios/rdstate/
C++ is not, and never has been, anywhere near that dogmatic.
As for the rest of your points, you seem to be arguing Rust is better than C++. But that's not really relevant to why C++ was rejected before Rust even existed. I do think Rust is even better than C++, but I also think C++ is a huge improvement over C, especially for large codebases.
However whereas you can write -fno-exceptions and get a C++ with this core language feature excised, you can't do the same for implicit allocation.
In contrast the Rust for Linux team have worked hard to produce a Rust for Linux environment that doesn't panic!() and has no implicit allocation, since these are red lines for Linus.
You're still trying to force C++ to be dogmatic. It never has been. It's always tried to be accommodating to all types of usages. It's always been multiparadigm. 'new' wasn't even defined to throw std::bad_alloc until C++98, which also introduced 'std::nothrow_t' to have it return null instead.
> you can't do the same for implicit allocation
Just like with exceptions this really isn't true, either. There's no implicit allocations in the core language, only in the STL. Saying that the "new" operator does implicit allocation is like saying "malloc" does implicit allocation. Regardless, placement new has always existed, and the non-placement 'new' operator has always been overridable. And all of the STL is optional & replaceable.
C++ has become lots of little silos of people using dialects that are in practice mutually unintelligible with other C++ dialects. Hence as I said the concern of the committee about this problem. What's the purpose of a "standard" which doesn't standardise anything?
But to be sure, Linus could create yet another silo.
And for implicit allocation, like I said, Rust for Linux did the work. C++ practitioners didn't. Linus isn't going to take something that doesn't exist just because its fans say if it did exist it would be suitable.
I don't follow this argument. In my experience, use of vtables is relatively rare in modern C++ development even when you aren't trying to avoid them, many of which are elided by the compiler anyway. Regardless, in these infrequent cases it is straightforward to code equivalent functionality without vtables, since it is syntactic type-safe sugar over the code you'd have to write in C anyway. They are like C bitfields in that every systems programmer knows how to implement the equivalent functionality manually for cases when the language feature is underspecified for the purpose.
Modern C++ without vtables is only marginally different than modern C++ with vtables, the code you'd write and the features you'd use would be nearly unchanged.
Between this and the other comment where you say Linus dislikes OOP I kinda get the feeling you ought to read some Linux kernel code. Because it's practically completely object-oriented and almost everything is done through a vtable for dynamic dispatch.
C++ would be a huge win all over the kernel, the only barrier to it is Linus' ignorance.
C++ has moved on from the days when that argument was made. This may have sufficed for an argument against C++ then but it should also apply to rust today which has similar infrastructure like generics and traits that allows a level of abstract programming that C doesn't provide.
C#, Java? Managed code with JITs, not supported on a kernel level no-way no-how. Ditto for Python, PHP, JavaScript, and other high-level languages without pointers and all that jazz.
Go would be close… but it requires bundling a bunch of libraries in with each binary and wasn’t designed for low-level development. Again unsuitable.
C++? Linus Torvalds and the kernel developers hate it for being unnecessarily complicated and only making things harder to maintain while bringing little to the table at this point (they’ve made it this far without classes, and C++ doesn’t bring anything new like memory safety).
Go has a garbage collector and doesn't allow for manual memory management which would rule it out entirely. This adds a requirement for the go runtime, which is a further nonstarter.
No it is not. There is Ada/SPARK and has been for a long time, and it is for critical systems. I recommend checking out https://blog.adacore.com/ for examples, there is plenty of that. For one: "An Embedded USB Device stack in Ada". There are also many articles about SPARK. For example: "Enhancing the Security of a TCP Stack with SPARK". These are just one of the recent articles.
For better or worse, the ship has probably sailed on Ada/SPARK making its way into mainline Linux.
People seem to love Rust. I've never seen that kind of enthusiasm for Ada.
Are there any non-proprietary implementations of Ada/SPARK which don't place limitations on the compiler output?
[0] https://news.ycombinator.com/item?id=24492653
[1] https://old.reddit.com/r/ada/comments/3989f4/gnat_gpl_restri...
Not for Linux, although they work perfectly fine in industrial installations running bare metal JVMs.
Rust is actually heavily influenced by C++, but enforces that those practices are respected as part of the language.
Where I'm less convinced that it's an option with a high likelihood of success is in open source. It's one thing when you've got a $250,000 salary riding on you ability to scrupulously follow best practices. But results may vary when the work is being done on an amateur basis.
In aerospace you probably want all this anyway, but for others, where 99.99% is enough, there needs to be a cheaper and more efficient way.
Sounds like a great way to have a toxic workplace that either yells at people for bugs, or never gets anything done because people are paranoid about committing bugs (or both!). Meanwhile, with a different language you get the compiler yelling at you instead. Automated enforcement uber alles.
I don't think open source vs commercial dev really matters that much.
Much, if not most, of Linux kernel development is done by employees of various companies, who are being paid for their work. Typically, that's in the form of device drivers, which also happens to be where I've seen the shoddiest, buggiest code in the kernel [1].
[1] Think "driver A abusing the 'private' pointer of driver B for its own purposes" kind of buggy.
2. unsafe blocks are explicit and therefore visible during code review. Memory safety violations in C++ are not always visible, simply because they are implicit.
3. Thanks to (2), unsafe usage in a codebase can be easily quantified and, if needed, audited.
Doesn't this assume that there are no bugs in the Rust compiler (for "safe" code)? Isn't that still somewhat of a big assumption at this point? I get that is indeed a very core goal of Rust, but has this been proven for any/all released versions of the Rust compilers?
Long story short, for the vast majority of use-cases, developers can assume that the compiler and stdlib are trusted. This is also how Rust defines "safety", which leaves you to focus on your code and its dependencies.
But "guaranteed" that there can be no security holes in code marked "safe" due to an inadvertent compiler bug? I would be very surprised if that is actually the case.
Less likely than C/C++? Yes of course. Guaranteed? Probably not.
Will the Rust compiler have bugs in practice? Of course. Do you - as a developer - have a different safety model than the Rust language (i.e., one that does not include the compiler)? If the answer is no, you can “ignore” compiler bugs from a safety perspective.
The harder problem is convincing people that raising their quality standards is a necessary requirement for them to contribute to the code.
Secondly, quality standards rely on the human element, which we know is fallible and never 100% reliable. Compiler-enforced memory safety checks are (in theory) infallible[1].
[1]: Infallible when all of your dependencies are also verifiably memory safe. In reality, this is not the case, but I believe that Rust can reliably get much closer to 100% than usual.
Addendum: yes, that last line is what pisses a lot of people off with Rust, and honestly, rightfully so. I've written quite a bit of Rust by now, and getting your code to work can feel like a bit of an escape room. Having C as an option on the kernel alongside it is a great idea though, because there are situations where I'd happily pick up C over Rust, and they're not just 'edge cases' where Rust falls short. Ideally, they're two different tools that can be applied depending on the situation at hand.
> With Rust, you actively have to fight the compiler to cause memory leaks.
Memory leaks are not considered unsafe in Rust[1]! They're actually an important part of Rust's lifetime semantics: intentionally leaking memory is both safe and equivalent to producing a `'static` out of thin air.
You probably shouldn't leak memory as a matter of practice, of course, but Rust will happily allow you to do so in 100% safe code.
Those are the most elementary rules that one must follow when writing C++.
Doing something correctly is not a best practice though. If it were, we could just say "if you correctly manage memory".
What happens when you mess up? What happens if you incorrectly model ownership with RAII? Or is it not really possible?
And what basic guarantees are not enforceable without ownership?
https://godbolt.org/z/re51ensMe
terminate called after throwing an instance of 'std::bad_optional_access'
what(): bad optional access
When rust was proposed for linux, controlled memory allocation was on the top of requirements list
Another good example is implicit comparison conversions, e.g. some class defines `operator==` as non-explicit and any code that does `x == y` where `typeof(x) != typeof(y)` may find itself going though a possibly-allocating constructor.
But even beyond that: you can have a non-trivial constructor that requires heap allocations internally. So even relying on (sane) compiler-defined behavior won't save you in the general case.
But that aside, it just seems like a very weird choice that would have profound perf implications - to the point of removing pretty much the sole remaining advantage of the language - for no gain whatsoever, since it's not even simpler.
As for non-trivial ctors - at that point you might as well worry that every function call made in pure C code can also heap-allocate, no?
Presumably the same is true of C.
C# has pointers and can be compiled to native code. I think the main problem is GC which is required for most of the standard library (you can avoid heap allocations but that would be unidiomatic)
AFAIK until recently (with the introduction of the Span<T> types), the only way to avoid heap allocations for arrays or any other kind of collections in C# was with stackalloc, which is unsafe.
Of course it doesn't make much sense to use it in the Linux kernel but it is a low level (or low level capable) programming language that can be used for systems programming.
The gap between Pascal with a couple extensions and C, or between Object Pascal and Objective-C, is very small, anyway. Most semantic elements have a one-to-one correspondence. There were even UNIX clones written in Pascal.
D I think suffered from a proprietary compiler and wide use of GC. But I’m not really familiar with it.
Exceptions are necessary to enforce class invariants.
While there are arguments over when and how to use exceptions, I think this is the first time I've seen them positioned as a killer feature of C++.
It's not a killer feature of C++, it's a requirement, without exceptions you can't have faillible ctors meaning your object lifecycle suddenly becomes a lot more complicated and full of additional traps.
I covered this option:
> your object lifecycle suddenly becomes a lot more complicated and full of additional traps.
because without ctors you don't have object invariants as you have to split instantiation and initialisation, or you have to implement bespoke and unusual strategies drawing you further and further away from anything which could be described as "standard" C++, thus making the advantages of C++ dwindle further into the distance. And all that with, as far as I know, no standard tooling to support you.
Classe invariants must be kept in each method call, not only constructors and destructors, and again they only matter in classes.
C++ is a multi-paradigm language, this is not Python Zen.
As a related but separate thought, sometimes adding invalid states to an object is useful. For example, take a look at std::thread::detach. Sometimes you see rvalue qualifiers on such methods (indicating the object has been consumed).
If you consider memory allocation to be infallible, there is a lot of stuff that can be written as pretty normal c++ without exceptions. The STL even pretty much works without throwing given this. If you consider allocation to be fallible you're going to have a bad time with "standard" c++ too.
And detaching a thread is pretty much always a bad idea.
The problem is not the ability to do it but the ability to enforce it. "Just be careful" has not worked, and will keep not working. Google's coding standard has not stopped C++ from causing them problems.
> As a related but separate thought, sometimes adding invalid states to an object is useful. For example, take a look at std::thread::detach. Sometimes you see rvalue qualifiers on such methods (indicating the object has been consumed).
You know what's actually useful and good?
For the object to actually be consumed and not be accessible at all anymore.
> If you consider memory allocation to be infallible
Which you can't in the kernel.
> If you consider allocation to be fallible you're going to have a bad time with "standard" c++ too.
Yes? That would be the point.
Disabling exceptions gives you C with classes, which is the old style.
I write ultra low-latency system-level code for HFT, and I certainly use exceptions.
Clearly, modern C++ has very little to do with exceptions, or I would have required them or found them useful at some point. I’m not saying exceptions are not useful, but their centrality to the language is empirically false given the vast number of modern C++ code bases that don’t use them and lose nothing by it.
If your process doesn’t survive allocation in bootstrap then it is badly misconfigured. If it survives bootstrap, then the software has been able to make hard guarantees about its resource constraints; the runtime will optimize to those constraints. Many years ago, given the designs of the time, you had to worry about blowing out the stack memory occasionally in edge cases, but even that isn’t a real problem these days due to changes in how it is done.
You would design things this way in any language. The benefit from a software design standpoint is that you can derive an enormous number of invariants from these constraints that can be used to guarantee that the core processes will never experience resource exhaustion. This eliminates a massive number of edge cases that would have to be accounted for if resource exhaustion was detected by testing limits.
Also when speaking about kernels, it isn't as if the other languages being discussed (including C), make use of their respective stdlib in kernel space.
Having exception-worthy invariants become panics would probably be a nightmare, at least for idiomatic C++: every nontrivial ctor (including copies and moves) might be a panic site.
The same argument could be made with return values: what happens if you don't handle them and just panic on them.
C++ also does not have decent macro system, which is leveraged in Rust error handling (the derive system).
Yeah macro like tooling in C++ is clunky, but on the other hand I think Rust goes overboard with macros.
I've notice that the trend of new compiled language is fairly recent with Nim (2006)/Go (2009)/Rust (2010). The previous trend was more oriented toward interpreted languages Python (1991)/Ruby (1995)/Javascript (1995).
So maybe, 10 years is the maturity age and we will see more of those language used for Linux kernel development.
Others, like Go, use garbage collection. That may not be universally unacceptable in OS kernels, but it's certainly not something anyone wants to see happening in the Linux kernel, which sees use in applications with low-memory and real-time constraints.
And the rest are dead or dying languages such as Pascal and Ada.
I can't speak to Rust's merits, I don't know it well enough, but if some reasonable subset of Rust makes it easy to stick to C-compatible ABIs without sacrificing most of the language's practical differences from C, then I would say that, yeah, it may truly be the first good - or even reasonable - candidate.
The biggest reason why Rust is, in my mind, a contender for the kernel is that it genuinely adds a new capability that C++ does not: true and guaranteed memory safety, with an unsafe core that is much easier to audit. Drivers written in Rust will be much less likely to contain memory safety problems, and that’s potentially a big win for kernel maintainers and developers. And yeah, it’s also interoperable with C to a large degree, which certainly helps.
Kind of a memory safety equivalent to what structured programming did for control flow.
I don't quite get this odd behavior of so quickly labeling a language as "dead" and devoid of any evidence to back up the claim. In the case of Pascal and Ada, various commercial interests want them to be "dead", because both represent a threat to their financial interests.
Pascal became Object Pascal, which is very much around in the form of Delphi/Embarcadero, Free Pascal/Lazarus, Oxygene/Rem Objects, PascalABC, etc...
Ada for sure is not going anywhere for many years to come because of its use in the military and aerospace industries.
Other candidates like Modula-2 or Ada, never had big sucess in UNIX space.
Objective-C never went beyond NeXT and Apple (yes NeXTSTEP drivers were written in Objective-C).
“I will not use C++ and programmers who develop projects instead of C are kicked out, lest they mess up the projects I’m involved in”, “C++ ended up with a bunch of terrible, unmaintainable garbage”
So how could C++ ever be possible in the Linux kernel?
The problem isn't technical, and to come back to your C++ example, there are plenty of OSes using C++ at the kernel level.
macOS being one of them via IO Kit, Windows since Vista, or that maker board loved by everyone, named Arduino.
Probably to Linus dismay, all Project Treble drivers (which run in userspace) on Android, are mix of C++ and Java, classical Linux drivers are considered "legacy" in what concerns AOSP project.
The only other language that could of had a chance of derailing the use of C on Unix and related was Pascal, because it was quite popular in many universities at the time and of similar capability. Consequently, people working for AT&T or connected to C development, viscously attacked Pascal in various papers and there was a lot of bad mouthing in the American media to help make sure it would not become such a threat. With AT&T so strongly pushing C and Unix, they got the result that they wanted.
Niklaus Wirth, the creator of Pascal, didn't defend his creation because he was preoccupied with Modula-2 and later Oberon. Even so, he was just a university professor, and any defense that he might have tried would have likely failed anyway in the face of AT&T money and connections.
If anyone's interested - here's Linus on C++ in the linux kernel: http://harmful.cat-v.org/software/c++/linus
Windows has been slowly migrating to C++ since Vista, when VC++ got kernel support for C++ code, naturally one just doesn't move something like Windows to C++ just like that.
https://community.osr.com/discussion/291326/the-new-wil-libr...
> First off, let me point out that this library is used to implement large parts of the OS. There are hundreds of developers here who use it. So unlike, uh, some other things that get tossed onto github, this project is not likely to wither and die tomorrow.
Also, both windows and mac has considerable and growing amount of C++.
Certainly the /more/ low level you go, the less C++-isms you see.
These days, sure you can use a highly restricted form of C++ in systems close to HW, but you might as well be using the industry standard in those environments, which is C, compiled with a C/C++ compiler.
-------------------------------------------------------
Language files blank comment code
-------------------------------------------------------
Rust 11813 339160 541958 2661720
C++ 9207 344592 198973 1632916
Go 2906 81163 138138 608871
C/C++ Header 7544 156294 209786 559658
C 1680 49614 52841 260405
Assembly 322 18112 11620 133536
-------------------------------------------------------
Header + C + Assembly 9546 224020 274247 953599This is untrue. Where did you get it? Have a look at kernel scheduler written in C++ here: https://fuchsia.googlesource.com/fuchsia/+/refs/heads/main/z.... constexpr! namespace!
Google has extremely strong internal coding standards. Those standards restrict which C++ features can be used, and how and where those C++ features can be used.
Without compliance based upon paycheck and performance reviews, I'm not sure that I'd want to try to make that kind of selective enforcement work.
If you look at the old code [1], it has a C++ extension & is likely compiled with a C++ compiler but the code itself is really just C. That's why cloc sometimes isn't an accurate tool for these. I don't have a sense of what the policy is but they could still be sticking with the C convention w/ C++ only utilizing things like constexpr & various other compiler pieces. They do seem to be making use of templates so it looks like there's been a lot more modernization done there.
[1] https://fuchsia.googlesource.com/fuchsia/+/9c7855e320662ad90...
You'd be surprised how many fortune 500 software companies have thousands of programmers who are writing new C code every day.
Even Apple writes new C every day.
C++ is also an evolving monster that everyone uses different subsets of, and many developers only know or use subsets of, either for lack of expertise or out of conviction (see how polarised the discussion of exceptions in C++ is here).
And every five years, even more complexity gets added, making the C++ story a modern PL/1-like story.
For many decades, there were also substantial differences in compiler maturity, forcing many to rely on subsets artificially.
At least C++ language features will have proper documentation, not this “if we place a struct inside this one at a specific place, we can use it as a linked list for no added memory cost” shit that one must know for each and every project (and is error prone).
People are talking about Linus' opinions of some other languages, but more importantly the Rust folks have been prepared to put in a lot of work to meet Linus' concerns. You're now seeing all sorts of other language advocates pop up with D, Zig, and so on, but they've not put in the same work to convince Linus that anything else are real options.
- C++ - Rust - Zig - Ada
I think Linus also commented that it has to be exciting and interesting for developers besides being useful obviously.
There might be objectively better languages for kernel development out there, but it doesn't do any good if nobody uses or knows them.
Rust has many qualities, but it's not a simple language.
The problem with complex languages is that they are ill suited for low level stuff.
Sometimes it's better to not have high level language features, because it allows everybody to understand the code without having to know every feature of the language.
Simple is better. There are too many developers who prefer complexity.
Certain problems have an inherent irreducible complexity. This complexity has to go somewhere. If the language doesn't tackle it, then users of the language will have to.
This is especially clear in the case of memory errors in C. C is too basic to express ownership of memory, valid scopes of pointers, and thread-safety of their usage. This doesn't make these problems disappear, only requires users to handle it manually and get it right without compiler's help.
In Rust you have to specify ownership and thread-safety of everything. It is complex, but not because Rust is complex, but because safe low-level memory management and thread safety are difficult problems.
Problem I have with Rust is that it uses certain patterns to make it work against the unforgiving borrow checker. I am not convinced that we will stay at this point.
Rust disallows some operations where the developer can know the operation to be safe, but the borrow checker does not. So you need to accommodate the language.
It's not a blocker in practice, because you can fudge things with `unsafe` if you know you're smarter than the borrow checker.
Sure it's simple in a sense there are not many keywords or features. And you can learn it over a weekend and start programming in it. But when you factor in undefined behavior, straight up user hostile choices done by common compilers in the name of performance, common patterns/library functions (that are all over books and stackowerflows) to avoid etc. C is not that simple anymore.
When you throw in concurrency, programming in C is quite hard.
I write this because I used to be an embedded C programmer, but that was over 10 years ago. I can still read code, sometimes even add simple patches (not to kernel ), so you could say I know the language. But it would take me several months of focused training before I would even dare to touch something as linux kernel.
Not saying that Rust is simple, just saying C isn't as simple as some people make it.
This is going to be unpopular opinion because it is more philosophical and completely unscientific.
I have been thinking something along that line for quite some time and not just on technology or programming languages. I think it has a correlation with age or how far along your journey in life. Simplicity has to click, it cant be told or learn just by reading. And more often that not, you can only understand simplicity after you try and fail to "fully" embrace complexity.
If you want a truly simple language you should have a look at Go.
Also, all of these are low level languages. There is no real difference between C and C++ in that way. C just lacks expressive power compared to C++.
I think we ought to use the new MAMAA.