System programming in Rust: beyond safety
blog.acolyer.org
blog.acolyer.org
It seems like the thing that happens most often is people complaining about "RiiR" and "RESF" etc, rather than the actual annoying things those refer to, so much so that it has become a self-perpetuating meme seizing upon the smallest hint of the bad behaviour.
Just experimenting myself, but I've had a positive experience so far.
Lots of great discussion on new tools, libraries, etc.
Bootstrapping Rust is hard, doing the same for C is simple. I could create a simple C compiler just to bootstrap the real C compiler, something I have actually done once or twice before. That's not something you do with Rust, at least from the limited experience I have with Rust.
What I don't understand is this whole thing about bootstrapping a new arch by building a whole new compiler on it, then compiling the source of an existing compiler on it (twice). Bytes of machine code don't have color, it's just a regular pile of bytes like everything else. Why does the compiler even have to run on the new arch? Just build a new backend for an existing compiler, then cross compile. It makes no sense to me why the machine code of a system must be assembled on that system.
I don't mean to sound adversarial, I'm just very confused.
Assembling the machine code of a system on itself means it's independent of its "parent" system. If I can't run the compiler on my new architecture (including compiling itself), I'll forever depend on having another machine with a different architecture to compile things. Yes, usually it's done by writing a new backend for an existing compiler, then using the new backend to cross-compile the compiler itself, and finally running the resulting compiler on the new architecture; you don't have to write a whole new compiler from scratch, unless you're worried about "trusting trust" attacks.
In the same way, when compiling a new version of a compiler, having the recently-built compiler build itself again makes sure it's independent of its "parent" compiler.
Far as a compiler, that doesn't really matter for most systems (esp embedded). You can cross-compile to any architecture you want with portable code and a new backend. The only tools you need for each architecture's CPU's/MCU's are something to load and test the code. Those already exist in embedded sector. Instrumentation for Rust code would be necessary but this doesn't involve doing the whole compiler on, say, an 8-bitter.
"you don't have to write a whole new compiler from scratch, unless you're worried about "trusting trust" attacks."
That's an SCM security problem. You'd need to trust every compiler developer, the repo it's stored in, the transmission process, and whatever you used to build it. Most supposed solutions focus on the last one almost exclusively when the first three were the source of most attacks historically. In any case, that's barely relevant to the discussion of using Rust on a new architecture as almost nobody will every be hit by that & most people wanting Rust backends aren't doing high-assurance security for stopping nation-states or something. Those people hand-verify the assembly anyway.
The first three aren't a "trusting trust" attack. With the first three, any attack is visible in the code that you eventually compile. The whole point of "trusting trust" is no amount of source code inspection will ever uncover the problem.
Plus, Karger pointed out solution shortly after the problem: verified compiler, safe languages, repo security (esp paper in safes), crypto/courier distribution, and designed to build locally from source using onsite tools rerunning all tests, proofs, etc. Mitigates the need for worrying about Trusting Trust most of the time plus knocks out majority of attacks. Better to mention that if worried about subversion. Or Myer's landmark work that defined in detail problems and solutions.
Trusting Trust meme just misdirects focus from important issues.
But the person who did said
> you don't have to write a whole new compiler from scratch, unless you're worried about "trusting trust" attacks.
The point of that comment (at least, as I read it) isn't that you have to be worried about "trusting trust", but rather that there's no point in writing a new compiler from scratch unless you already are worried about "trusting trust" (with the implication being that most people aren't and so writing a new compiler from scratch isn't necessary).
SCM security issues (as you mentioned) are irrelevant at this point, because they really have no bearing on whether you have to write a new compiler from scratch.
[1]: https://www.ieee-security.org/TC/SP2016/papers/0824a018.pdf
So that leaves just independence, which you solved right in your comment: use an existing compiler with a new backend to cross-compile itself. If your compiler is portable this still means the only thing you need to write is the new backend.
Maybe one written in C89 that has its own backends for various archs. It would be less-focused on optimization and more on portability.
x86_64-mscv
x86_64-osx
x86_64-linux
armv7-android
i686-android
arm-linux-hf
armv7-linux-hf
But there are literally billions of 8bit and 16bit embedded systems out there, many of which have a C compiler.
http://www.atmel.com/Images/45107A-Choosing-a-MCU-Fredriksen...
http://www.embeddedinsights.com/channels/2010/12/10/consider...
Far as future, Jack Ganssle of The Embedded Muse has a nice assessment of it summarizing various sides:
https://juliacomputing.com/blog/2016/03/10/j2c-announcement....
I have high hopes for something like https://github.com/thepowersgang/mrustc
C wasn't that portable in the 80's and 90's, with all its flavous across CP/M, MS-DOS, 8-bit home micros, mainframes, ...
Writing a rust compiler in assembly might be challenging.
https://www.reddit.com/r/rust/comments/6h5mht/system_program...
Not a wasted question after all given prior, positive feedback on those projects.
Your interpretation may vary.
Also, now this thread is better than the one over on Reddit.
IMO rust is a replacement for C++, not for C. C is much closer to assembly then most all other languages, especially those that deal with memory allocation natively.
I'll note that Ada developers have been writing high-performance or real-time code in C's domain for some time without relying too much on unsafe pointers. They mostly use references or restricted pointers. It has additional benefit of separating types from how many bits represent them where miles and kilometers might be 32 bit but compiler error if you mix them. Also, bounds-checks on arrays or pointers. Then, SPARK subset that verifies absence of common vulnerabilities has no pointers at all. People still manage to build useful things like the IRONSIDES DNS server.
http://www.adacore.com/knowledge/technical-papers/safe-and-s...
https://en.wikipedia.org/wiki/SPARK_(programming_language)
http://ironsides.martincarlisle.com/
These show the advantages of C's pointers are overstated in the general case. Actually a drawback if building high-integrity software. Worst case, we write just those few algorithms in C, unsafe Ada, unsafe Rust, assembly or whatever then wrap them in strongly-typed, safer functions with input validation when called. That's standard in safe languages. Probably how Rust people do it, too.
It's just a different set of tradeoffs.
Rust's raw pointers have equivalent power to C's pointers.
Not native, see C.
>Rust's raw pointers have equivalent power to C's pointers.
Again, AFAIK you can't do type punning.
That wasn't even my point, that was that C gives you almost everything a cpu does and not much more.
Bitflags crate is native; 100% Rust, 0% C. Here is the source: https://github.com/rust-lang-nursery/bitflags/blob/master/sr...
But i give up, as this isn't a technical discussion.
To be honest, I don't know if a functional difference, but I have not done any embedded programming where there might be a difference.
Now, I never did it that way myself. I just and-ed it with a bitmask to get the part that I wanted. But it's a bit messier to do it that way, because if you want one of the fields as a number, and it's not from the least-significant bits, then you have to shift it as well as mask it, whereas with bitfields you can just read it as a number and get the right value.
Never bothered me on Turbo Pascal days, plus if I remember correctly bitfield ordering is not portable across compilers anyway.
I suspect it's the same with type punning (though also, to some degree because we're still working out the details of what exactly the semantics of unsafe are.) In the end, you can always cast stuff to bytes and then cast those bytes to something else.
I don't take (too much) objection to that, and little with "not much more", but that doesn't mean that C has some secret stuff that only C can do, which it feels like you're implying.
I don't know how that macro works, but in kernel C bitfields are used in structs that map to device registers (that are mapped to memory).
>I suspect it's the same with type punning (though also, to some degree because we're still working out the details of what exactly the semantics of unsafe are.) In the end, you can always cast stuff to bytes and then cast those bytes to something else.
So if i want to store a series of things with random (length) types in a single array, i can ? Sounds like i can, even though you explained the simpler case.
>I don't take (too much) objection to that, and little with "not much more", but that doesn't mean that C has some secret stuff that only C can do, which it feels like you're implying.
And for the third (edit:second) time, that is not my point. Thing is that C does not have "secret stuff" and rust does. In C you get what you wrote, and not much more. There is no need for telling the compiler to not do some checking or to not make a copy of something. (edit2: granted, bitfields are not native to cpus as type punning is)
Rust, IMO, is more for the C++ crowd then C. And those two are very different.
This is really not the case, not with modern C compilers. You may find calls to functions from the standard library to be converted in surprising ways, while touching undefined behaviour can eliminate evaluations, branches, function calls and more. Your mental model might have a close resemblance to assembly; but compilers don't see it like that any more.
Yes, things like memcpy and printf are (in most cases) built-in in compilers. Not surprising really.
Undefined behavioral is bad programming, if your ask me. Almost all examples of it, that i ran across, are in really ugly code that only a "smart" programmer would write.
Clangs static analyzer points out undefined behavior, and gcc is close to getting a "-Wundefined" flag that points that out at compile time.
A optimizing C compiler can change the resulting code even more then just doing dead code elimination. But the point still stands, that C maps well to assembly and that languages that do things like their own memory allocation are not even comparable to what the cpu provides (only automatic memory allocation that a cpu somewhat provides is the stack, and C exposes even that).
Commonly-used type punning techniques frequently exhibit undefined behavior due to strict aliasing: https://blog.regehr.org/archives/1307 .
It might be true but barely matters when we have tools like KCC. It's built on an executable semantics of C that runs in a framework on top of Maude rewrite engine.
http://www.kframework.org/index.php/Main_Page
https://github.com/kframework/c-semantics
http://fsl.cs.illinois.edu/FSL/papers/2015/hathhorn-ellison-...
Same thing. There's no requirement to use it in a kernel context, though you can if you'd like.
> So if i want to store a series of things with random (length) types in a single array, i can ? Sounds like i can, even though you explained the simpler case.
Yes, though it'd use a lot of unsafe code.
> And for the third (edit:second) time, that is not my point. Thing is that C does not have "secret stuff" and rust does.
Ah, sorry! The contrapositive (or whatever) is tough. My bad for misunderstanding.
And no-no should program in assembly unless they know exactly what they're doing.