Rust: Systems Programming for Everyone
infoq.com
infoq.com
The whole point of Rust is to give you compile-time errors (as opposed to UB or runtime checking) when you are doing something unprovable and force you annotate your code so that things become provable. That involves understanding up-front a lot of things I learned the hard way while writing in C and C++.
If I just want to quickly get something done that involes hitting the bare metal, then C is probably easier than Rust. What Rust makes easier is getting up to a high standard of confidence in the coide. But easier is still hard.
1. I am re-writing the book with the knowledge we've gained watching people try to learn Rust over the last 18 months. For example, compare https://doc.rust-lang.org/stable/book/ownership.html to http://rust-lang.github.io/book/ch04-00-understanding-owners... ; The former version takes a rules-based, theory-heavy approach. The new version uses a concrete example with Strings, a common thing that people new to Rust struggle with.
2. Error messages are being overhauled across several axes https://blog.rust-lang.org/2016/08/10/Shape-of-errors-to-com... IDE integration is also being worked on.
3. We created a dedicated "beginners" channel on IRC, it's been a rousing success so far.
and hopefully more things in the future.
> If I just want to quickly get something done
I especially like the error messages and the compiler's ability to tell you how to fix your error. And though I don't like to be dependent on IDEs, it would be great to see one which integrated that guidance into the editing.
But all of these are still palliatives around the fact that Rust is a complex and difficult language. Rightly so it's rules embody a lot of the practical wisdom of highly experienced programmers and lets them (and sometimes forces them) to express their advanced ideas.
> numerics
You can find higher confidence in code being correct in other programming languages. "provable" is better used to describe interactive (Coq, Isabelle, etc.) and automatic (Z3) theorem proving languages. Interactive theorem provers generally require a large amount of work on the coders part. Automatic theorem provers are limited in the types of problems they can prove. I am sure they can still run into runtime issues.
But like all type-checkers, it does prove that your code obeys certain rules. In a trivial way this is even true of C -- but languages like Rust and Ada go further and prove that you are avoiding large classes of common bugs.
If it was, in every sense, the right term, there would be a lot more fuss around that.
> Type checking (safety) is only one subcategory of proofs
I hardly know what I'm speaking of, but these lemmas on nLab [1,2] seem to imply more than that.
[1] https://ncatlab.org/nlab/show/propositions+as+types
[2] https://ncatlab.org/nlab/show/relation+between+type+theory+a...
> Type checking (safety) is only one subcategory of proofs
Type checking as provided by Rust, Haskell, etc. only covers a subset of possible proofs. Liquid Haskell goes further to increase the range of possible proofs it covers through type checking by using automatic theorem provers.
I'm pretty pro-Rust, but I spent some time last night trying to write up some fairly trivial code in it. I hit the complexity wall very quickly. Because I don't use the language often, I have to relearn the core concepts of borrowing, etc. anew each time. It was quite frustrating and I kept twitching thinking how easy it would be to fire up CLion, write a CMakeList and write what I was doing much more quickly in C++11...
I've been programming in Haskell and Python in the last years, but I've done more of a decade with C++ only and lately begun some MCU work where I had to use C++ almost exclusively again. I don't have a strict preference anymore, although I edge in Python for new projects as it strikes a good balance.
I'm quite familiar with runtime type checking (hs), and hey, I've actually wrote a couple of programs in ATS (whose syntax is absolutely horrid, and second only to the compiler warnings), because ATS essentially translates to C without an extra compiler, allowing you to use it for any MCU supported by your toolchain.
I came out thinking that doing seemingly simple stuff in Rust, such a mutable linked list, is too complicated for what you get. Let's be honest here, writing a safe linked list in C++ (even more so with C++14) is _trivial_, despite C++ being anything but simple to understand as a whole.
This is aggravating if you think that in systems programming, mutable state is almost central to performance (caches and locality). Most of the efficient structures you end up writing are ad-hoc, mutable and with manual memory layout. Complicated code is a double edged sword: if I need to write complicated code to get compile-time insurance, it means that it's a trade-off to just jumping into unsafe territory where the payoff is small.
I somehow like the syntax of rust, but it's quite verbose. Almost too much in places. And I compare this to C++, which is not too skinny either. But C++ got much better lately. I also really don't like how rust inherited the JS style of wrapping closures directly as arguments of functions (think unwrap). It really reads horribly. I'd have vouched for dedicated syntax for such a common idiom. C-like languages (and C++ included) do not read as nicely as homoiconic languages in these cases.
If writing simpler code leads to less bugs, then Nim is actually a much better language. I wish it had more mindshare, as the core and language itself is far from being polished and regular as rust currently is. The current premise of compile-time safety of rust is good, but not a dealbreaker if you use a modern C++ compiler IMHO. It's pretty easy to wrap unsafe behavior in a class. It's actually easier than wrapping unsafe code in rust.
This is really interesting to me as a curious outside observer of Rust. I conclude that either writing a safe mutable linked list in C++ is really not easy or Rust's type system makes it too complicated to capture simple invariants and properties.
Which is it?
The trivial "safe" C++ linked list prevents you from dereferencing uninitialized memory, leaking resources and performing incorrect type conversions, which are the typical issues you'll find in any C linked-list implementation. But there's no compile-time check for the actual lifetime of the objects. This means you can manually destruct the same object twice, for example, and the compiler will not warn.
RAII can get you /very/ close, but since it's not enforced at the compiler level, Rust guarantee is definitely stronger.
My main point is that the strong guarantee comes also at a significant complexity cost, and I'm still not sold it's worth it in many conditions.
I can roll with it. It's probably worth it. I'd love a job programming in it. But it's not entirely the dream I had.
So have people at Mozilla (or people anywhere else) and there are still horrible bugs in Firefox or other similarly large projects.
> is too complicated for what you get
If you aren't convinced by arguments that safety is intrinsically valuable for yourself or your customers then another argument is that with safer languages you pay the price upfront (i.e. learning and programming with what seems a more complex language) instead of later spending weeks or months on catching a bug causing crashes.
Rust is not on the front of research here, although the borrow checker is more pleasant to use compared to /many/ other implementations.
Memory safety is, of course, important, but security as a whole is so much more than just pointer dereferencing and lifetime checking. We got pretty far with C++, modern toolchains and OS support. Rust programs will be only marginally more secure by default than other classes of programs. Rust will not help with entire classes of other problems like side-channel attacks, any logical bug (like incorrect initialization of IVs, unchecked reflection), and so on.
Rust helps, and I do not criticize that. But as with any system, you have to evaluate it as a whole. Is it a big boost coming from C? Absolutely. C++? Not so much. Heck, you can use LuaJIT in embedded systems, and Rust has zero advantage there.
I think my knowledge of haskell helped immensely in learning rust though, since many concepts appear to be lifted from haskell. the records system is similar, the Option type is similar to the Maybe monad, Result is similar to Either. they both have algebraic data types, there are type aliases, there is a way to declare newtypes with unit structs, traits are similar to typeclasses, etc.
It seems obvious in retrospect but if you can check syntax there is no reason to leave it for compile time. The danger is that you put time into things that you have to go back and change because you didn't know when something went wrong. I think the same thing can be said for borrow checking and any other static analysis integrated into a language. If it is left to compile time it is a guessing game with a lot less interactivity.
Of course some would say that that mentality is just for beginners, but we've heard the same thing about IDEs, syntax highlighting, auto completion, etc., and historically anything that makes programming easier and takes mental load away from a person has been a huge benefit.
There's a good talk by Ander Hejlsberg about how it was designed from the ground up to support things like syntax highlighting and autocompletion, on the fly; as I understand, even syntax highlighting can be done by asking the compiler for the AST, as opposed to regexing, and it's all computed and cached as an incrementally built pipeline.
I remember using Eclipse for Java development years and years ago, which has an entire incremental Java compiler built in. As a result, it offered lightning-fast error display and autocompletion, because it never needed to compile more than exactly what you were editing. It was amazing, and ever since I have been disappointed that this approach did not catch on elsewhere -- either people didn't know about it, or they chose to ignore it.
Even Go -- whose toolchain is very much designed to be run programmatically by apps such as IDEs -- seems to have dropped the ball there. There is an autocompletion daemon (https://github.com/nsf/gocode), but it's not part of Go.
The rust compiler basically taught me how to write safe systems code, that's an amazing thing. Even for trivial programs, if I attempted to write it in C, it would be buggy and unsafe.
It did no such thing: as one example, if there's an error in your model, would the compiler catch it? Maybe, maybe not.
Safe systems code requires a proper software development lifecycle (SDLC). Rust could be a part of that SDLC, but alone is insufficient to produce a safe system. Advertising it as such is common, but I think very misleading.
I don't know why you would think I meant 'safety' in some other way, considering Rust advertises that it is a 'safe' language, and they are obviously talking about memory safety.
It seems like you just wanted to construct an easy strawman to knock down.
As for beginners, I simply can't believe that C can be easier than Rust, when C's idea of memory management is basically malloc() and free(), with everybody having a custom implementation, not to mention the gazillion string implementations, all broken and incompatible, along with having nothing resembling a portable memory model.
Sure, it makes sense in the context of the Linux kernel, but in the user space the fact that C and C++ are still used is a tragedy.
C/C++ aren't "for everyone" because everyone has been hearing for decades that they are terrible languages and should only be used for certain kinds of things by certain kinds of people. True or not, that's basically what they've been hearing.
Nobody is saying that about rust. The message is more like "go for it, and tell us what you did; the compiler will help protect you (and your users) along they way".
If rust isnt easy, I think that's OK. Not ideal, but OK. Lots of people have goals that they're willing to work for.
It seems to be a multi-paradigm language like C++. I guess it's advertised for system programming because it's looking for its niche, but I don't see it replacing C for low level stuff due to the learning curve.
For example, C++ Core Guidelines [0] is longer than Rust Book. Core Guidelines is itself a great example of things one needs to keep in the head while writing C++ code.
Look at implementation of something as basic as optional<T> [1]. The amount of things one needs to know is so high most C++ developers would never attempt it.
[0]: https://github.com/isocpp/CppCoreGuidelines/blob/master/CppC...
[1]: https://github.com/akrzemi1/Optional/blob/master/optional.hp...
That's actually simpler to do in Nim than you'd think, but I understand what you mean. I might expand my implementation to be const and see how complex it makes it!
C++ has a massive amount of accidental complexity. To mention just two: the difference between () and {} initialisation (for some types) especially in the presence of perfect forwarding, and a set of feature/their interactions that means there's a bug in the following very reasonable looking code (can you find it?):
std::unordered_map<T, U> map;
// ... initialize map ...
const U *selected = nullptr;
for (const std::pair<T, U>& p : map) {
if (condition) { selected = &p.second; }
}
// use selected
Rust may seem more complex because the compiler is noisier, but it is significantly simpler to write correctly: a computer is finding the weirdest bugs.
You are right regarding the C++ accidental complexity though. Issues like the one you highlighted tend to happen when working with pointers. They are very hard to detect during code review (I didn't see it), but I would have rejected the code at review anyway because it should be replaced by std::find. Taking raw pointers into containers is also not reasonable except for vector.
> I would have rejected the code at review anyway because it should be replaced by std::find
That specific example, sure, but there's other ways to have escaping pointers in for loops.
> Taking raw pointers into containers is also not reasonable except for vector.
The map types are explicitly designed to allow taking pointers, e.g.
> References and pointers to either key or data stored in the container are only invalidated by erasing that element, even when the corresponding iterator is invalidated. -- http://en.cppreference.com/w/cpp/container/unordered_map
How many C or C++ programmers will understand this: (1..n + 1).fold(1, |accum, x| accum * x)?
Anyway, I am just offering my opinion as someone doing mostly C++ programming. I have no problem with a language like Swift or Go, but feel that Rust is too complicated for what it offers.
Taking a pointer into containers refers to &p.second. We now have a pointer to an iterator's member variable. That is unreasonable.
The reasonable thing would be to use an interator to keep track of the selected item, this is what they are designed for.
The fix to avoid this is to use auto. If instead of pair, you used auto&& p: map, this would not be an issue
Probably too early to say if Rust will become the lingua franca of system-engineering. I'd love to see them succeed and hope that it isn't trying to be everything for everybody. We already have enough of those.
Personally I think TUF[1] is still too academic and at least for now doesn't offer practical solution to this yet.
[0] https://blog.valbonne-consulting.com/2015/04/14/as-a-goat-im...
[1] https://github.com/rust-lang/crates.io/issues/75
EDIT: typos
A lot of folk on Dockerhub are just folk. They don't have security teams keeping an eye for relevant CVEs and triggering rebuilds.
I work for one of those companies (Pivotal) that does have a security team keeping an eye out for relevant CVEs. I work in buildpacks. If you send an app to a Cloud Foundry installation, you used the rootfs image and buildpack binaries that we built. When a high-value CVE lands we commit to providing replacement bits within 48 hours. We usually do it in much less.
But even with extensive automation -- and ours is pretty extensive -- this requires humans to watch the firehose of flaws. Most people who write a Dockerfile and forget it don't have that luxury.
Incidentally, Docker now offer security scanning of dockerhub images, for a fee.
This is weird. I thought it would be in their best interest that dockerhub's images would have better reputations.
And, in fairness, it's a surprisingly tricky problem which many well-heeled customers see as worth paying for.
At Pivotal we've been working on "AppDog" to do something similar: inspect running applications on Cloud Foundry, tell you what dependencies are installed, what their licenses are, whether there are updates available and so on.
And yeah. It's harder than it looks. There are edge cases upon edge cases.
(Of course, nothing I say should be considered official comment etc etc).
> Docker Cloud and Docker Hub can scan images in private repositories to verify that they are free from known security vulnerabilities or exposures, and report the results of the scan for each image tag https://docs.docker.com/docker-cloud/builds/image-scan/
The feature IMO which make rust really great, usually only get mentioned in passing : well define compile time environement and compilation process,fast compile, sane macro system, ML like type system with pattern matching, great tooling and ecosystem out of the box.
> forgoing the advances made in C++ and Dlang
(Allocators are the asterisk here, those are still a work in progress.)