Microsoft: Rust Is the Industry’s ‘Best Chance’ at Safe Systems Programming
thenewstack.io
thenewstack.io
Personally, I think the article is somewhat correct. Well vetted and tested C (or C++) is great, but even the guys who are looking for these things get it wrong sometimes.
Rust isn't perfect, and I think the negative reaction is people thinking that evangelists consider it perfect. But I certainly wouldn't call it perfect, and I'm rather fond of Rust.
That said, I still think it's the "best chance", that doesn't mean it's the only possible thing to make headway into more safe systems, but it means that it has the 'best chance'..
Maybe you can make D memory safe, but when I was getting into D development I couldn't even get a compiler working.
Maybe Ada is your favourite, but I don't know anyone writing Ada and integrating it with a C codebase.
--
Frankly it comes down to the fact that Rust was designed specifically for replacing components of a large C++ codebase that it will be adopted. If you can write a driver that works with linux that's written in rust and you don't have as many memory safety issues or concurrency problems (and theres no overhead) then that's reasonable.
Even if Rust never gets supported by the kernel's buildsystem officially or if the code can not be upstreamed.
I'm a Rust user, a fan of the language, and I believe it's a great step forward for systems programming.
However, it's kind of ridiculous how on any thread about Rust, anyone who brings up criticism, no matter how valid, gets downvoted into oblivion. Just look at what happened to the comments critical of Rust on this post.
The Rust community really needs to take care not to become an echo chamber.
The whole "RESF" thing is not in any way welcomed by Rust community and governance folks. It's not funny, it's harmful, and anyone posting it is doing a disservice to the language they purport to support.
To anyone reading this: please don't downvote constructive criticism of Rust. Save your downvotes for the comments that wish Rust wasn't as welcoming to everyone. ;)
The thing about constructive criticism, when it comes to something like programming language design, is that its usefulness varies inversely with the importance of its aim.
Core features of the language are highly important (obviously) but criticizing them is not very useful since they're highly unlikely to change. Thus, only the most peripheral and superficial parts (as well as new additions) are subject to valid, constructive criticism.
People who have criticism for (or are otherwise skeptical of) the core parts of Rust are not being constructive, then, and so these fights can break out.
I'm fairly certain it's a meme.
The only counter example that I can think of is the way that the actix unsafe maintainer's behaviour was handled (brigadiering bands of Reddit evangelists flaming him for his own flaming). However, I chock that up to an anomaly caused by Reddit's toxic community and not representative of the Rust community.
Within the Rust reddit or forums, they specifically dissuade blind evangelism and there are often posts critical about the language that get quite a lot of traction.
Outside the main rust community, there are definitely zealots but they are usually not accepted by the larger community, and are usually called out.
I’m not saying it’s perfect, nor that there aren’t zealots who go overboard. However it’s probably one of the least toxic of the programming communities I frequent (swift, c++, Python, c# , go etc)
1. If the tech hasn't made full contact with reality yet. For instance, there aren't a diverse set of large projects written in rust yet, so it's a little easier to imagine that it's perfect.
2. We should acknowledge that technology choices have a big impact on one's career. Sure, we are all software engineers that could work in a lot of environments; but we are simply more productive in the ones where we have some experience/depth. If someone is learning rust today, in many respects that's a bet, and they (subconsciously or consciously) have a vested interest in rust's success. More experienced people will usually have a more balanced risk portfolio, spread between proven tech and a few up-and-coming technologies. But for people earlier in their career, it might be a pretty major bet for them.
It's kinda like the old military policy of "don't ask don't tell" - the policy wasn't actually don't ask don't tell - the actual policy was "pretend to be straight". Guys didn't get in trouble for talking about their girlfriend.
Beyond that, as someone that doesn't care about politics, I just skip the thread. There are dozens I don't care to read about. It's just another one. No big deal.
Think of it instead as "people who are political" looking for a community (not necessarily because of that community's politics, but often for orthogonal reasons, e.g. a language being relatively new and so not yet used by anything they dislike) and then those people being political about their programming wherever they land. If some percentage of programmers are like that, then just statistically, some language is going to end up with a lot of them, whether its maintainers wish to cultivate the community image as being such or not.
By this implied definition of political, more or less any human social interaction is political in the broader view.
Maybe you had a particular bad experience after someone mistook your behaviour as trolling - not that you would have been ... I just haven't witnessed any bad behaviour in the community yet - either towards myself or others.
If anything, I have appreciated at how much support the informed members are giving to the relative newbies like myself. Not only in advice but listening without ridiculing if the speaker is not that knowledgeable on the topic of conversation.
@jasondclinton, I agree with you in regards to the bad experience the actix project owner got in regards to unsafe code use. It was sad to see that happen to an open source contributor (or anyone).
When I made this comment there was nothing I could call "valid" criticism.
There are valid criticisms though; Lack of dynamic linking and immaturity (and most crates only building in the nightly channel, and not easily knowing if there was a version buildable on stable) are things I can think of as large weaknesses.
I know these and I'm not a full time rust programmer.
It's true that you can mate up to C by exporting the C ABI everywhere. This isn't a free lunch though. There are annoying and/or weird side-effects you sometimes have to deal with when leaning heavily into doing this.
It is possible to keep the ABI backwards compatible by practicing good C hygiene, but many library authors doesn't know how. Especially when it comes to Windows which is a platform most C library authors are not intimately familiar with.
Exactly. The language has been proven to be stable, however a valid argument is the unsoundness risks of a project when they begin to depend on lots of third-party crates.
The moment one imports a crate or third party dependency into their project, unless the dependency explicitly specifies the lack of unsafe{}, all bets are off on the 'safety guarantees', especially when 'that crate' is used in multiple projects.
As for an echo chamber, yeah the Rust community downvotes way too much. I find myself upvoting just to counter it.
But the downvoted comments here are legitimately bad. Someone being pedantic about the 70%, and being wrong, and another person calling this propaganda.
Many people in the Rust community appear to be unaware of this. Which is actually a big deal. Many of the bugs with the most catastrophic consequences are caused by over and underflows. Consider for example a gas pedal indicator kept as an unsigned that underflows causing the indicator to reach its maximum value.
If safety was my primary concern, I wouldn't choose C++ but I wouldn't choose Rust either.
Well, it depends. If the unsafe{} code is also unsound, then "all bets are off" is quite correct because you can't assume memory safety or lack of UB, even in your own code. In practice, you can use cargo-geiger to warn about crates that internally rely on unsafe code, and cargo-audit will warn about known security issues (including but not limited to unsoundness and memory-unsafety).
I'm not going to argue you should use rust or not use rust but I think the state space is not binary and people reading will benefit from that.
Can't cargo refer to arbitrary git repos or even just plain directories? I don't see how it's any different from C or C++ in that case.
do you just mean reddit and hacker news posters because I find calling that part of the community pretty absurd that's just lowest common denominator internet yelling
IMHO, on an ideal platform for debate, the audience's experience of the debate should itself resemble the statistical distribution of opinions held by the debaters.
I.e., if 80% of people think X while 20% of people think Y, then comments saying Y should not make up 90% of the debate by any metric (number, word-count, above-the-fold representation, etc.) If Y did take a majority share by any metric, that'd be a misrepresentation of the debate, that is likely to cause the audience to come away with a misunderstanding of the issue.
I don't know how close we are to an "ideal platform for debate" on HN (probably not very)—but I feel like a certain degree of people reflexively downvoting contrarian opinions (no matter the subject) probably bring us closer, not further, from that point. Contrarian opinions should not be presented as if they were orthodox opinions. They should be given some representation—I'm not advocating for censorship—but they should only be highlighted in any algorithmic way, to the exact degree that they're recognized as something some appreciable group of people really believes.
Rhetoric works to win debates because debates are time-limited (and the audience's attention is both time- and attention-limited), and so you can win just by e.g. preventing your opponent from ever getting a word in edge-wise, or forcing them to answer for trivial side-issues and therefore never have time to say what they really want to say; etc.
What I'm railing against here is the https://en.wikipedia.org/wiki/False_balance: the rhetorical rearrangement of non-equal evidence into a debate presented to the audience as if each side had equal support behind it.
And, make no mistake, that sort of rhetorical rearrangement is what discussion forums with comment voting are doing when the community rearranges the comments with upvotes/downvotes: they're curating which comments appear first (and thus which comments appear at all, to those without interest in reading the whole page.) They're changing the impression of aggregate weight-of-evidence someone gets by skimming the debate.
But my claim is that this is not a bad thing, when it is done in the goal of presenting the debate accurately. If one side of a debate has little evidence, but more voices, those voices should be rhetorically damped down to compensate, until the force of their combined voice matches the volume of the evidence on their side. That is the proper goal of editorial curation in e.g. news journalism: to let authoritative primary sources speak loudly, secondary sources quietly, and kooks and rumormongers not-at-all. And it should, IMHO, be the proper goal of a community's editorial curation of a comments section, as well.
This is a natural consequence of the restrictive rules imposed on the rust community.
Can you be specific?
> Respect that people have differences of opinion and that every design or implementation choice carries a trade-off and numerous costs. There is seldom a right answer.
> Please keep unstructured critique to a minimum. If you have solid ideas you want to experiment with, make a fork and see how it works.
I am not "anti-CoC" but I do see how the particular wording of this one could be interpreted to silence pretty much any technical discussion that someone doesn't like.
That's just not true.
The review process and standards maintenance must be very strict.
Otherwise, as someone said, "A C compiler is a mechanism to turn coffee into security advisories".
Yes, I'm a Rust moon-boy ;)
Sure, because you people talk about it literally everywhere.
> other one is the fear of realizing that Rust is going to take over some day.
It is not. It is a good systems language, not so much for application domains where other languages get shit done in half the cognitive load. just because rust community likes to promote rust for everything and involves in dishonest marketing that borrow checker is not a problem, rust is not the optimal language for applications domains where high guarantees and low resource usage aren't mandatory.
I've still to hear why Rust is better than Ada and I caught the tail end of that madness.
With more static analysis baked into the language itself although, comes the trade off of more restrictions and longer compile times.
However, if you can get past the startup barrier, D is a real pleasure to work with.
That is not how the (Linux) kernel likes to work…
As far as "how the Linux kernel likes to work" either it's in tree, or it doesn't matter. (Not my opinion per se, IDC, but look at upstream's collective treatment of ZFS and other out of tree drivers in general; they remove APIs they know external modules are using). I do generally find out of tree drivers to be a PITA to build outside of the kernel, and sometimes find curious things in their build scripts. Updating NVIDIA drivers seems to be...not great.
"here" meaning where? Can you point to it, because there's not "a lot" on this thread.
So to evaluate whether it's worth introducing rust, you'd ideally want to find a self-contained part of the kernel that is responsible for many CVEs and you'd want to show a substantial reduction of CVEs with the rewrite of just that component. Not impossible, but also not so easy.
Perhaps the way to do it is to rewrite common bits that get CVEs, just so it doesn't happen again.
[0]: https://c2rust.com
EDIT: Oops, read the parent comment backwards.
We can look at actually statistics instead of just one random anecdote. For instance, according to [1], there are 2 million lines of rust code in firefox, 6 million lines of C++, and 3 million lines of C. Very roughly speaking that's 20% of the lines of low level language being in Rust.
Firefox only started requiring rust to be built about 3 years ago, this is a pretty breakneck pace of replacing C/C++ with rust.
[1] https://www.openhub.net/p/firefox/analyses/latest/languages_...
While Intel/AMD have put incredible effort into hyperthreading, virtualization, predictive loading, etc, page faults have more or less stayed the same since the NX bit was introduced.
The world would be a lot safer if we had hardware features like cacheline faults, poison words, bounded MOV instructions, hardware ASLR, and auto-encrypted cachelines.
Per-process hardware encrypted memory would also mitigate spectre and friends as reading another process's memory would become meaningless.
[Mostly discussed deeper in the thread already but felt it was worth bringing up directly as a reply to this]
This is an active area of development though still plenty of work to do. The Cambridge University Computer Lab have been doing research in this area in the form of CHERI: https://www.cl.cam.ac.uk/research/security/ctsrd/cheri/ this gives you hardware capabilities which are effectively pointers with bounds given out by the OS. Want to access something? You need the appropriate capability to do so.
ARM announced MTE last year, which whilst far less capable than CHERI begins to give you something (though it's targeted for bug hunting effectively rather than providing any actual security/safety properties): https://community.arm.com/developer/ip-products/processors/b...
ARM are also building a hardware CHERI implementation, Morello: https://developer.arm.com/architectures/cpu-architecture/a-p.... There's already CHERI implementations running BSD on FPGAs, this will take it to the next level with real silicon using modern ARM processors (maybe you could run android on it for instance?).
I think Intel have been looking along similar lines but I'm far less in touch with what they're up to so I don't have links.
This isn't really true. The main performance cost of these safety checks (and largely of others such as overflow checking) is inability to optimize because the compiler needs to preserve partial results/states in case a fault occurs. The checks themselves are trivial.
I want efence on steroids.
I think I was having trouble envisioning what exactly a "cheaper" check could look like in hardware. Bounds checking is basically read length, subtract your index, and a conditional branch (potentially with a hint that it would succeed).
To do this properly in hardware I suppose you'd need a list of memory regions that are "live" and default the rest to "dead", though how many do you support? What does updating the list look like? Page tables are pretty slow to update, and those don't change too often. Array tables would be pretty gnarly, and impose a further penalty on context switching as they'd have to be thread-local and app-local.
I wonder if this is a case akin to spinlocks. Sure, I'd love a lock that doesn't busy-wait, but there's not really a cleaner solution -- in hardware or otherwise.
Maybe I'm just not seeing something obvious, though!
Sure, the world would be a lot safer if we used microkernels, too, but the tech world has been obsessed with performance over all other characteristics for decades.
At first, I just thought it was odd to make the hardware more complex by having two address spaces. However, this prevented a common cause of difficult to find bugs in asm programming, and I came to appreciate that hardware could make programming safer.
Hardware architecture, programming languages, and compilers advanced rapidly, but safety always seemed to take a backseat and was left up to the programmers. I’m glad to see developments like Rust and I look forward to using it for a real project soon.
So make use of Solaris SPARC, iOS or Android 11 (ARM HMT is a requirement).
Solaris SPARC, iOS and now Android 11, all make use of some kind of hardware validation in memory accesses.
Sorry if my question is really stupid. My experience has been with scripting languages, not systems programming. But I read the chapter in the Rust Book about Ownership, and I think I get it. In a sense, is not the compiler simply inserting function calls to free() in the right places (and warning you if it can't)? Couldn't this be added to the C compiler? In fact I'm not sure what the difference is from a linter.
The article has the phrase "C/C++ Can’t Be Fixed", so it anticipates my question. But its answer is that programmers either won't remember or won't bother to set their C to "strict mode", run the linter, or whatever, because "static analysis comes with too much overhead: It needs to be wired into the build system. . . . If it’s not on by default it won’t it won't help." But if this is the only argument, it seems weak. I agree that it is more effort, but isn't it much more effort to switch out your entire language and ecosystem?
Sometimes. There's a lot more to safety than memory safety, but that's the easiest to define.
Other aspects might be safety against null values, safety against poor error handling, etc. Those are much harder to define.
> Could not the C compiler be given the same thing, with a flag like "strict mode"?
Not really, no. There's a lot of research into proving C code safe, and automation of it, and it's not really feasible for general C codebases. There are tons of static analysis tools that try to get part of the way there, but they can be slow, miss things, and have many false positives.
It would require, at minimum, some seriously ugly annotations. There are papers that demonstrate such annotation based approaches with C. One that I read ended up looking quite a lot like Rust, at which point, you wonder why you wouldn't just write rust.
Not a stupid question at all, by the way.
> Couldn't this be added to the C compiler?
Rust is able to insert those frees because it has lots of information about memory usage at compile time. C doesn't, therefor C can't insert those frees in the general case.
Other issues, cultural ones that is, like "people just disable it" are fair but not the biggest issue imo. But to further emphasize that point there are many runtime mitigation techniques (ASLR, stack cookies), and they get disabled all the time because, and this is the extra sad part, they legitimately find bugs and crash a program that otherwise may have appeared to work, and the fastest "fix" is disabling it.
Re: Effort, totally, yes. But the effort isn't really split. There are tons of people working on safer C/C++ through tools like fuzzing, static analysis, and runtime mitigations.
Because it is another language.
If the researchers behind the borrow checker had implemented it on a safe variation of C/C++, then it would have been much easier to push for it in many companies. Something like the Python 2/3 split is better than another completely different language.
Don’t get me wrong, Rust brings improvement in other areas too, which is fine, but when we are talking about many-million-LOCs, you don’t care that much.
I am still waiting for someone to bring lifetime annotations to C and C++. Those I could use right away. Rust, not so much.
I tried to port an (Earley) parser from C to Rust. It wasn't even a port, but more of a rewrite with similar function structure. The first part worked well, even though I had to resort to an arena (a special type of allocator that can guarantee lifetimes at the expense of freeing them later than strictly possible). Then I tried to get in shared parse trees. It was a nightmare. It could simply not be done with the same efficiency as in C. I had to resort to using array indices instead of pointers, or use ref counting where it wasn't needed. This had quite some impact on the calling functions.
And that was a few hundred lines of plain, non-tricky C.
There will be places, no doubt, that you cannot do it, but if you can easily cover 80% of the code, that is way better than nothing and then you can work on redesigning the 20% (perhaps even rewriting it in another language since it has to be changed, for instance to Rust). Same argument as Rust uses for unsafe blocks.
There have been specialized annotations for things like mutexes for a while in several projects which help static analyzers prove things too, so it is not new.
And at that point why not just use Rust. It doesn't require you to switch out the entire ecosystem, you can happily link Rust and C, and C++ together in one binary. In fact, as an end-user it's sometimes easier to use a popular C library from Rust because someone will have wired it into Cargo (Rust's build system) and it will "just work": you don't have to mess around with working out what kind of build system the library is using and integrate it with whatever your using.
See my sibling comment. Using Rust implies many more changes than just moving code to an incompatible-but-close version of C and C++ that adds lifetime annotations and unsafe blocks. Compare it with the Python 2/3 split vs. rewriting all your Python 2 code to, say, Ruby.
Then there is the OOB issues too, like using another build system, a single compiler based on LLVM, etc.
Even moving large enterprise codebases from python 2 to python 3 is taking so long they had to push that deadline back years. Given the distance you'd have to cross to push an older C++ codebase to something with similar guarantees as Rust, I don't know if any company would voluntarily cross that gap.
Worth noting that the vast majority of lifetime annotations in Rust are “elided”, i.e. inferred by the compiler.
I’d say that a successful “safe C” compiler would accept any existing C code that never invokes undefined behaviour, without a runtime performance penalty.
That's unfortunately what's not possible. The C source code doesn't contain enough information to prove safety (a lot of real world C code depends on invariants that happen to be true at runtime, but aren't provably true at compile time)
Rust has an affine type system [0] and this is something you either have or you don't. There is no way to retrofit it without rewriting all C code in existence.
When you look at linear types it becomes pretty obvious how restrictive they are but this restriction is also what allows them to make guarantees about memory safety. With linear types every variable must be used exactly once. This means that no matter through how many functions you pass a variable eventually you have to call the destructor to destroy the variable.
Rust doesn't have linear types because they are too restrictive. Instead it has affine types where variables are used at most once and if they are not used the destructor is called automatically. However even that is too restrictive. The definition of "use" in Rust is a borrow and the affine type system rules only apply to mutable borrows. You can have an arbitrary amount of read only borrows. The end result is a language where it's not possible or much harder to implement data structures that do not follow these rules such as doubly linked lists. You will have to use completely different data structures in a variant of C that has an affine type system. This will probably result in an affine only ecosystem that is completely disjoint from the rest of C. The Linux kernel will definitively never be rewritten in affine C.
Mind you, I don't know the formal theory enough to say whether you can build a static analysis tool that is actually capable of doing what Rust's lifetimes accomplish in a provably complete manner. Maybe someone else can chime in here.
However, there are other disadvantages to this approach. Not having it in the language means everyone working on the program has to be using the exact same static analysis tools, and they have to be using it on the entire program, including libraries. Rust ensures that the lifetimes are enforced consistently, since there's only one possible engine enforcing them.
There are some very impressive code analysis tools used in the industry (with C and C++, in particular), but far as I know, they are all commercial and fairly expensive. As far as I know, open source tools like Valgrind are not powerful enough.
Take a look at some of the system utility tools on crates.io, such as ripgrep (like grep), bat (like cat), fd-find (like find), lolcate-rs (like locate), procs (like ps).
What difference does that make in the real world?
Embedded systems frequently have flash size limitations in the megabytes or even kilobytes.
You can certainly compile rust to fit on there without any trouble though, you just have to try... the same way you have to try to compile any language in that sort of environment really. You can get to about 30kb while still keeping the standard library [1], you can get something with a fully featured web server in a few hundred kb [2].
And that's fine. You wouldn't run most C cmdline tools on an embedded system either.
This is a fairly low bar for correctness, and not really one I would really agree with :/
The same logic can be applied in Java or C++ or probably even C, but it also comes down to the wide spread and idiomatic approaches of the language. The rust eco system is not necessarily coherent around the details of errors, but error handling is generally rigorous. The benefits of that can’t be overstated. The compiler is a huge help in getting it right in a way that Java checked exceptions never was.
I’m sure that there are other languages that are showing similar wins, though. I haven’t written much Swift, but I imagine that the experience might be similar?
I am no fan of rust and actually hate its evangelism. But this is ridiculous.
Deleting the production database is a thornier philosophical question, but Rust's position is that it is.
In Rust I feel like almost every time the program works on the first time it compiles successfully, even without writing tests. Probably a combination of very well designed API, builtin RAII, very explicit syntax, compiler strictness and clear docs. I also cannot forget to handle errors (even if I just decide to propagate or crash).
This is what "correctness" means to me, but it isn't exactly enforced by the compiler itself.
> Microsoft c++ continues to be written and will continue to be written for a while
I interpret this as saying that they cannot instantaneously convert all their existing code to Rust, and it seems fair enough. However, it seems a little odd to start new projects in C/C++ when you are publicly saying it's the wrong thing to do.
Infact, just a little over a month ago they announced a new QUIC implementation written in C/C++ (MsQuic)
Is this to be expected since Microsoft is such a big company? Or maybe that lib is not important and is not going to be used in, say, Edge?
Moreover, isn't networking code the worst kind of code to write in C/C++ (I am thinking of heartbleed for example)? I would really love some explanation.
https://en.wikipedia.org/wiki/Manu_Cornet#/media/File:%22Org...
They are quite clear that a constraint version of C++ still gets to play in a secure world.
And while Azure Sphere still gets to be written in C (due to Linux kernel), it has a security layer (Pollonium) that takes care of bad code.
Finally, since they failed to move people into UWP programming model, sandboxing has been coming into Win32 as well, to the point that on Windows 10X, everything is sandboxed, including Win32 apps.
There are many gaps in my understanding of Rust with regard to LLVM and GCC.
The rust compiler generates some intermediate code (LLVM IR) that's fed into LLVM, which optimizes and turns it into machine code. You can check out what this looks like in the rust playground. Getting GCC to compile rust would be a matter of getting the rust compiler to generate GCC IR.
Cranelift (built with rust) is an alternative to LLVM with it's own IR. The cranelift backend for rust should help future projects that want to do something similar, I guess.
Systems programming is not web where they change the framework and 'script flavour every six months.
Systems programs live for decades - and hence their critical infrastructure like GCC will as well. C will be with us at least 3 decades more. Longer if Linux kernel will use it still.
That has already began. For example all the popular browsers (Chrome and all the derivatives, FF, Safari) moved away from GCC.
My questions is where to find jobs with rust. I’m really enjoying it, but my resume is heavy python/react atm
I’d be (negatively) surprised if AWS started from scratch instead of from Rusoto. That was the approach taken for the original Go SDK, which was adopted from Coda Hale’s work.
I migrated it recently to async/await and the code was much easier to read.
Could have done it in Python's boto3 or any other SDK. Rust does not give a performance benefit since it is mostly waiting for AWS answers but that was an opportunity for me to just write some Rust code.
Edit: for people looking for rust work in the Bay Area, you can message me. Email is username @gmail
As an advisor on these kind of projects I usually advise clients to be very conservative about the tech used by 3rd party developers or agencies..
no seriously. you are right. however that never had been a problem. mostly weird langauge choices also attract pretty capable developers.
I know that is not what people mean with "unsafe", but it is getting tiresome.
Even parts of managed languages which are typically frowned upon from a "systems programming" perspective, such as GC, can be viewed as a positive when trying to iterate. GC can be treated like a backstop which helps to keep your application from exploding while you work on memory management, and then you can incrementally address heap alloc (aka latency) concerns with an application that actually runs (for a little while at least). Trying to iterate on memory pressure and related concerns with a C/C++ application is nothing short of nightmare fuel from my perspective, but I spend most of my time in higher-level languages these days.
I think it is also very hard to make a performance argument in 2020. Especially when you are comparing low level against something like the newest C#/.NET Core stack. Just take a look at the most recent TechEmpower plaintext benchmark (round 19). How many new business "systems" are being "programmed" that aren't ultimately just web servers passing HTML/JSON around, executing business logic, and reading/writing bits of data to some database?
I think it is very EASY to make a performance argument in 2020. In every domain, from my phone, to my desktop pc, to our huge AWS servers at work, I encounter software that is slow. We struggle with our C/C+ database at work running on top of a C/C++ operating system. We can't buy a bigger server and it isn't fast enough to handle the load we get recently recently due to covid. You think I should be happy to incur GC overhead in both the OS and Database? No way! That would cost us a lot of money, and require a more complex architecture. (which also costs money)
On my desktop and phone I encounter applications that are annoyingly slow to start up, to compile, all kinds of things. The last thing I want is JVM or .NET runtime overhead occuring even at the level of my operating system, device drivers, etc adding to that. CPU cache space is precious, the low level things that everything else runs upon should be as efficient as possible.
C++ fired back because integration was ugly (the SDKs are designed for C usage) and a lot of C developers that doesn't understand C++ well.
Go is used a bit, but again, training people for using it correctly is hard. And using the C SDK from Go is an obvious NO.
A lot there think that Rust would be in a similar situation that C++. Nice to have, but too much low level dependencies (hardware SDK) and high level (developers proficient in the language) to expect it to happen. Also, calling the SDK would convert everything I'm unsafe, so we are like in C, but with a language that nobody understands.
So, there are technical reasons, but also human resources and economic ones.
A complete Rust system would likely help solve most of these issues and drastically reduce these bugs, increasing development speed. Unfortunately the lack of native SDK in Rust is annoying, but some groups are making progress on STM and no-lib Rust. It’d be great if STM makers adopted their sdk and made an official one!
Currently since Rust doesn’t support my chip, I’m using Nim to generate C. It doesn’t solve all the data races noted above but having generics and easy data structures like tables and vectors is fantastic and I'm not second guessing the code all the time. A deterministic GC covers my memory usages fine.
Also, those high level languages require an OS. And many systems programs run on bare metal or a very minimal OS.
Edit: Also, the abstractions that higher level programming languages provide are often not really needed for low-level programming. In some ways, low-level programming is easier than application programming, since the actual programming required is often fairly straightforward and constrained by the application.
C# (per your example) is not orders-of-magnitude away from from C++ perf like Python is, but is w/in a very small constant factor, and often even faster for poorly-written C++ as GC is more efficient than calling copy c'tors where they needn't be called, for example.
And if you introduce features such as Span<> (and friends) and ref returns into your C# where you need them, you now how have safe GC-free/reduced code with perf that rivals Rust in the sections of code where you need it, with full GC as a backup.
One thing I like about Rust is the idiomatic way is usually very close to the fastest way. There are exceptions of course. But it you tend to get steered to efficient solutions.
With that said, there isn't much of an excuse for non-0-cost enumerators, etc. -- given that refactoring tools switch between foreach, linq, indexing or ptr/spanning, you'd think the compiler-chain or jitter could do the same. Roslyn is a better back-end for meta-programming, but it's too bad MS denies us the benefits of surfacing meta-programming to the language or at least tool level, or we'd have tools that could easily convert linq to highly optimized code.
You could also have compiler-enforced "levels" of C# code that had restricted features/semantics, such as "alloc-free", etc. - right now the choice is just "default" or "unsafe".
"LOH allocation detected for object 'MyNamespace.MyObject'. Attempted allocation bytes: 90K (maximum 85,000 bytes)."
Then one level up from that restriction would be the totally alloc-free approach you mention which could potentially mean the GC could be put into a special idle/framework-only mode. For alloc-free, I believe you could enforce this at compile time, as opposed to run-time for the LOH condition.
The performance overhead of a virtual vs non-virtual call on the CLR is negligible - same as with C++. For proof, look at Direct3D which is performance-critical, yet Direct3D’s API is based on COM which is all about vtable-calls.
Linq itself is also very decent. I won’t argue that Linq is perfect (e.g. suboptimal list allocations when the total output length can be known in-advance), but its hardly “slow” - certainly not enough to impact any production workloads - you’d only observe a difference between hand-written loops and a Linq expression in a contrived synthetic benchmark.
It isn’t. It looks like COM, but it isn’t.
Long-term: I feel that Swift was meant as Apple’s answer to C# and as way to retain XCode users as Objective-C became increasingly unfashionable over the past decade and Apple lost employees who could properly maintain it. Swift won’t go away, but I suspect that Apple doesn’t want to have to pay to maintain Swift while the rest of the industry gets to use it for free and port it to platforms that Apple won’t see any revenue from: i.e. Apple doesn’t want to be like Sun/Oracle and have other companies (Google) use their work (Java) for a competing platform (Android’s Dalvik).
So, I’m wary of adopting Swift - while Apple does currently support it on non-Darwin platforms right now I fully expect Apple to lose interest in maintaining first-class support for Windows and Linux at some point, just like how OpenStep was repurposed as a Mac-only when it became the rational decision for them.
Maintaining a thriving developer ecosystem and platform requires a degree of trust, transparency and a history of commitment to a product - and Apple (as a company) is the anathema of transparency.
However I think you are correct in that they won't want to put much effort into it. Their support for developers is already somewhat half-hearted if you compare it to Microsoft. Using XCode over the last few months has made me appreciate how good Visual Studio is. Apple's developer documentation has me missing Microsoft's.
[1] https://play.google.com/store/apps/developer?id=Apple+Inc.&h... [2] https://support.apple.com/en-gb/HT210384
It seems that it's not available both for the largest desktop platform, Windows, and for the largest non-desktop platform, Linux on ARM (https://swift.org/blog/swift-linux-port/ says "Currently x86_64 is the only supported architecture on Linux.") i.e. all the IoT devices which need code in a "systems language" and Android phones.
This is not a good approach in general. GC is only really needed in applications when working with data that's graph-like and potentially involves cycles of object references. A software project should be properly designed at the outset so that GC can be used in a "pluggable" way where it's strictly needed. Rust will probably enable this kind of use in the future, but other languages such as Go can already be used to comparable effect, seeing as they're generally limited to smaller pieces of code ("microservices") running in their own separate address space.
Fully agree with you that GC should be abstracted away and used for where it shines: cycles and graphs!
- Not leaking memory - Not invoking unsafe - Not angering the rust compiler
None of those efforts stabilized to my knowledge, and the libraries developed for this task all have experimental/research tags associated to them.
For better or worse all the guides for using rust strongly encourage avoidance of primitive cyclic structures such as doubly linked lists, graphs, and other structures in favor of statically allocated fixed size variants. Like golang's preference for map/list as the only primitive structures, this is a constraint that one can get used to for most problems. But it doesn't feel great that as a language there are basic algorithms and datastructures that can't be idiomatically coded in the language.
Still extremely new, of course, but has promise!
IMHO "systems" here means running on an OS, not network systems or internet apps.
So None, but you want to write libraries that are easily linked to all manners of 3rd party languages and can squeeze the last drop of OS performance. Your language with automatic garbage collection cannot do that. If all you end up doing is wrapping stuff with C or C++ headers then better use C or C++ at first place.
So I would say there is definitely at least one GC language that is worth using for systems level programming.
I agreed until this. I don't take anything not AOT compiled with an optimizing compiler as fast.
JITs may work well for numeric benchmarks with one tight loop, but there is extra memory and energy overhead.
Also, I’ve read traits can be used as invariants for a pseudo-dependent type system. Has anyone had experience with this?
I’m considering learning it instead of C++ for some systems work.
No, this isn't true. https://blog.rust-lang.org/2018/07/27/what-is-rust-2018.html... was the last change and it was optional and automated migration.
Rust doesn’t have a stable ABI but the API is non breaking as long as you aren’t on nightly AND opting into unstable features.
If you stay on stable Rust, you’ll not really have breaking changes between versions.
Regarding ABI , that means that if you compile rust into a library with one version, you can’t reuse that compiled library in a newer version of the compiler, unless you go through a stable ABI like exposing a C interface.
However if you recompile your code, it’ll work. You need to keep them compiled by the same compiler version.
I would be interested to know if this is really the official opinion of Microsoft, or if someone just wants to make a name for himself.
You mentioned Ada, which does have a minimal runtime. It might be interesting if the community and ecosystem made a strong push into general-purpose programming, like rust has. I would give it another try.
Rust seems to handle heap allocations more fluidly (or at least closer to what I expect), but perhaps that wouldn't be a deal-breaker if I knew more about progamming in Ada. Rust also has better C integration -- it can match C structure layouts and operate on them seamlessly with rust semantics. But basically, it seems plausible that Ada could work with the right ecosystem around it.
Regardless, Ada has had decades to make the leap into general-purpose programming, and it hasn't done so yet. Rust is viable today for a lot of important use cases, and is not far off for a lot of others.
Swift, Zig, Rust, Go, etc. are popular because there has been a resurgence of new languages in recent years and a lot of new, young developers are helping develop libraries for them.
It is simply not as fashionable to program in old languages so some new developers don’t.
> And just as one can write secure code in C++, it will turn out in time that even with Rust not everything is as secure as propaganda would have you believe.
I think the point is more that Rust is memory safe by default, and you have to opt out of it. In C++ on the other hand, memory safety basically falls to the developer, with plenty of crazy pitfalls to watch out for in the language. It's not that it can't be done, it's just harder to do it right. Rust makes the easy, default thing safe, and the unsafe thing requires extra effort.
Yes, unfortunately; the trend seems to go in the opposite direction; but anyway: if I really wanted to switch to something new and spend the effort to invest in new tools and education, then I would rather choose a technology with a decently long track record demonstrating that it fulfills all my expectations because of which I left my original technology. Rust will take one or two decades for this; Ada/Spark has already been through this.
What I hate is rust fanboys' propaganda that rust is great for every purpose including CRUD stuff because crab god moral and borrow checker is moral responsibility. There is lot of dishonest marketing sounding like rust doesn't have cognitive overhead compared to other languages. It gives me an impression that, in best case, it is selection bias among people that successfully learn and use rust, or in worst case, some people trying to show off that they learned a 'hard' language and it is 'easy' for them.
Secondly rust community is very political and polarising that many people don't like. It almost seems full of non-STEM-worthy people although they are a vocal minority..
An example from a while back: needing to rewrite your whole original Ruby code base in Scala. Or having to write your own PHP compiler.
TL;DR it has some support; however most libraries are just C bindings, or else they are very domain-specific.
You can, but it is not as smooth as using other languages.
[1] - https://github.com/microsoft/verona
[2] - https://github.com/ponylang/ponyc
(edit: formatting)
I love Rust, but I'd refrain from such sweeping statements until other options evolve and get the opportunity to be time tested.
Maybe providing/funding a different back-end would be a worthwhile endeavor for Microsoft to do or fund.
It has a great toolset, generates good code, and with a good microkernel architecture (which Windows 10 is, mostly) 90% of "systems" programming would be fine in a garbage-collected language.
I know, I know, efficient, separate compilation and parametric polymorphism are hard to get together, but a systems language should not follow the same compilation model as, e.g., Haskell.
There is a trend, and the trend has been a MASSIVE reduction in memory errors. The percentage of security issues that are memory errors says nothing of the totals, which is what matters.
Microsoft could go through 2021 with only one CVE and we'd see articles like this saying "100% of security vulnerabilities are caused by memory errors, we must switch to Rust!"
The magnitudes here are in the many hundreds of CVEs per year from Microsoft (and growing), not "one CVE in 2021". 70% there is not a negligible number.
[1] https://github.com/microsoft/MSRC-Security-Research/blob/mas...
Except that CVE's seem to be increasing over time across the industry, not falling. There is no overall reduction in the numbers.
I think this is more of a "make an idiot-proof system and the world will make a better idiot" type of situation.
While I think that unmanaged (i.e. no runtime or interpreter) memory-safe programming languages are a good idea, they're not magic and we shouldn't ignore the other major security tooling improvements recently.
Citation?
What is the evidence for this? I'm not familiar.
Developer installs continue heavily invested in dangling pointers (called paths) and other unsafe practices, C/C++ style. I find that baffling.
So I installed the full package, 4.6GB and now it works.
I guess Microsoft isn't being helpful, despite Rust being their best chance ;)
Yes, I know Rust is not a panacea, but I feel like the whole paranoia around safety/security is just leading us towards that sort of society, technology included. It seems that there was somewhat of a balance before, which allowed for some sort of "civil disobedience" (jailbreaking, rooting, cracking, etc.), and now that freedom is slowly disappearing.
Yes, more secure code does reduce the chance of jailbreaks, but insecure code is overwhelmingly used more by criminals and state actors to try and break into your computer.
Secure Code helps all computer users, the 'little guy' most of all.
And you can always vote with your wallet (or, you know, vote) to keep your digital liberties. Blaming a better tool for eroding your liberties seems ... strange.
In a way, making code more secure is just creating an asymmetry where regular joe can't break into a device but only state actors and other major players can.
I can sort of understand GP's sentiment. If someone really wants to hack me, they will. Or they'll find some other vector. The prevalence of memory safe code doesn't really impact my own security as an individual. But it does impact my ability to jailbreak and own my devices.
There was a similar discussion around the intel Trusted Computing concept. It was not a platform that ultimately benefited the user, it was a platform for user-hostile forces to 'trust' that the user isn't interfering with the code they want to execute on the user's device.
Slowly but surely we're moving towards a world where user devices are merely thin clients over corporate and government owned systems, and don't really belong to the user at all.
Today, when macOS gives me warnings and asks permission for inane things, like terminal needing permission to look into ~/Downloads or ~/Documents, I and everyone else with some experience in technology just rolls their eyes at this request and move on. This would be like if my new toothbrush asked me 'are you sure you want to brush your teeth in this bathroom?' However, to someone not familiar with technology and Apple's security logic of recent years, these requests might seem terrifying, and scare off what could be bright minds who could have contributed to the field of computer science had they not been questioned at every turn by their operating system of their intentions. You learn by breaking things, after all.
Complexity is the problem, hard-to-read code is the problem, over-engineering is the problem.
It won't solve the problem, but it can be an important step towards a solution.
> Complexity is the problem, hard-to-read code is the problem, over-engineering is the problem.
I actually disagree. Many memory safety issues are not the result of hard-to-read or over-engineered code, but are instead due to careless memory management. Safe Rust code (no matter how hard-to-read or over-engineered) can guarantee memory safety.
My bet is that Rust won't help nearly as much as people tend to think.
They tried many things in the past, including C# and .NET, I'm not sure if that really helped or made things worse.
Which other languages close to C, without a run-time, and without a garbage collector, give you proven memory and thread safety?
Not C, not C++, not D, not Nim, not Go, not Swift, not Ada, ...
FWIW, I suspect C with proper tools is fine, just fine.
Also Ada is still a thing.