I'm really worried that I still hear about buffer overflows in this day and age. Of all the libraries in the world, glibc should probably be written in some subset of Idris that compiles into 100% safe C. We have the technology to move to this now
This is important if you export multiple symbols for a function, that is, have a program that depends on a specific implementation of a function. That could mean you depend on different versions of the same library ("functionname@@VERSION_1, functionname@@VERSION_2"), or completely different implementations of a function ("functionname@@IMPL_X_VER_1", "functionname@@IMPL_Y_VER_2").
Wait a second. That seems like a strange feature to want, right? Well. It's particularly useful for runtime library fun stuff. LD_PRELOAD in a library that magically changes the implementation of function. A cool example of this is of preeny[1], a collection of libraries to help with debugging.
Additionally, it's useful for replacing libraries that would depend on C implementations without having to recompile them, though some preloading magic at runtime may be required.
This is only the thing, for me at least, that's blocking for making Rust system libraries. Now, any sane person may say, "Cargo will take care of versions and stuff", and that's true. Sadly, that's not the case if you ever want to use those nice and safe Rust libraries from C though.
Though, I may be completely wrong about the state of symbol versioning in Rust. I simply ran 'nm -g libopenssl.so' on sfackler's rust-openssl project and failed to see any @@<version> in the output -- that is specifically being exported, and not re-exported, by libopenssl. There may be better libraries to use, but eh.
I could be completely wrong on this though. I've been mainly going off of what I read here[2], and here[3] (alternatively, for a more in-depth version, try here[4]).
TL;DR: symbol versioning may not be a thing.
Disclaimer: I've had to use symbol versioning pretty heavily, so it's important to me. I could also be completely wrong about Rust not having it.
[1]: https://github.com/zardus/preeny [2]: https://gcc.gnu.org/wiki/SymbolVersioning [3]: https://www.akkadia.org/drepper/symbol-versioning [4]: https://www.akkadia.org/drepper/dsohowto.pdf
As far explicitly using the symbol versioning scheme you mentioned, you can do it with an attribute today:
https://play.rust-lang.org/?gist=92be155702d53c6411b2&versio...
(Disclaimer: I haven't actually compiled this as a dynamic library and checked, but its what that attribute is for, and the names seem to be used verbatim in the ASM output)
It seems as if symbol versioning in Rust is nicer than in C. I'll have to play around with that a little bit.
I'm hoping I never to have to deal with writing version scripts or inline assembly again -- or at least having to write less.
Are you seriously suggesting that rewriting an implementation of the C standard library in a language that isn't C is something that makes any bit of sense?
I think you have a fundemental understanding of the roll the C-standard library plays in the C language. Not to mention UNIX in general.
Maybe you're suggesting we need an OS based in Rust instead of C (which is more reasonable), but in that case why wouldn't you use Rust's standard library as the backbone of that system?
Let me put it this way, if the GNU/Linux C standard library gets rewritten in Rust or Idris (sigh... seriously?) I will switch to Windows 10 with default settings for the rest of my life. I would rather have the NSA log my every key stroke than participate in a community where that is considered a good idea.
Wrap existing C with safe Rust, not the other way around! We don't rewrite unsafe C code in Unsafe Rust so that it can be called from all the unsafe C code in UNIX. Such a translation is so pointless and error prone.
I can't even believe I'm the one being called backwards here.
They're definitely not. The power of Rust is not avoiding all `unsafe` ever, but wrapping that unsafe into finite-scope, safe wrappers. Something like mmap would presumably interface with the OS's internal memory management routines, which would definitely have to have `unsafe` somewhere since they're hitting the hardware, but it seems very reasonable for the amount of unsafe exposed to much reduced (possibly even to zero). See, for instance, http://os.phil-opp.com/modifying-page-tables.html . Basically, as soon as you're above the absolute raw hardware level, you can start introducing extra help to reduce unsafety/catch bugs.
In any case, code inside `unsafe` still benefits from all the conventional Rust checks, e.g. iterating over a slice won't go out of bounds due to a typo in the loop even inside `unsafe`, nor will references accidentally become dangling. (Of course, the `unsafe` block may do something explicitly that causes either of these problems, but this exact same risk is pervasive in all C code, rather than just around explicitly marked areas.)
> Wrap existing C with safe Rust, not the other way around! We don't rewrite unsafe C code in Unsafe Rust so that it can be called from all the unsafe C code in UNIX. Such a translation is so pointless and error prone.
Note that this is exactly what is done now, the features of Rust for making safe interfaces makes Rust often a far nicer way to use C interfaces than C itself, without overhead (IMO, of course). See http://blog.rust-lang.org/2015/04/24/Rust-Once-Run-Everywher... , for example. Of course, zero-overhead Rust is only as safe as the C code it wraps, it can't (in general) protect against the C code not correctly implementing its stated contract(s).
Maybe you could separately map shared RefCells to guard the pages returned by the mmap wrapper? If the region is backed by a file, you'd need to check those RefCells on every IO access too, though.
Examples would be:
* read-only: &ROMemMap -> &[u8]
* read-write: &mut RWMemMap -> &mut [u8]
* write-only: fn set(self: &mut WOMemMap, i: usize, b: u8),
or more generally: &mut WOMemMap -> &mut [WriteOnly<u8>] where WriteOnly<T> has fn set(&mut self, T)
Now for the shared case, consider this: aliasing rules can be avoided with atomic operations, i.e. Arc<AtomicUsize> is shared and can be safely (atomically) read/written by multiple threads.
In the multi-process case, you could provide an atomic API, although we don't currently seem to expose byte-level atomics (likely not present on some platforms) so if you wanted to write a demo you'd need to use the unstable intrinsics atm.
FWIW restricted to single-threaded code, this results in the Cell get/set API which is not hardware-atomic but cannot overlap with other accesses to the same memory, as Cell doesn't implement Share so any threading abstraction will block you from doing any kind of sharing of Cells.
That is, you can share &[Cell<u8>] pointing to any bag of bytes that sits in read-write memory, and anything in the same thread can read or write to it, safely.
However, as you say, exposing a safe mmap version for libraries to call is a different problem with different difficulties. eddyb's sibling suggestion of teasing out the multiple entangled uses of mmap sounds like a good one, although someone would have to experiment with it to be sure.
Cool stuff though, I really hope rust catches fire soon. I love that it has revitalized the appeal of implementing operating system level projects like text editors and shells. I never would have wanted to try and write a windowing system before Rust. Now that's something I'm actively investigating for fun.
There's no technical reason why the C standard library can't be written in a language other than C. Lots of language libraries are written in languages other than the one they're primarily used with and work fine.
There's no technical reason why it can't be done, true. It's still a hilariously bad idea that would never succeed. That's aside from the fact that's a completely pointless wheel-reinventing endeavor. Rewrites in general are pretty difficult to justify.
Sorry, I guess my aggressively pragmatic side is showing today. The idea is really dumb.
To be honest, "Let's rewrite the C standard library in Rust" is one of the worst pieces of bikeshedding drivel I've ever read. And I like Rust 100 times more than I like C.
The way forward for C on Windows from Microsoft own SDK is to use the clang frontend + Visual C++ backend, or another C compiler from someone else.
Yes, C++ is as unsafe as C, given its copy-paste compatibility, but at least it provides the tools to program safely (if one chooses to) and the C++ community cares much more about safety than the C one.
... and together with stuff using glibc (indirectly) constitutes 99.99% of your software.
How much software/languages implement their own DNS resolver instead of just calling libc? How about syscall wrappers?
A managed language generally doesn't give you direct access to syscalls at all. In rust you can perform them without having to go via C.
One possible reason why java might implements its own name resolution is because it wants to do async name resolution which is not possible (at least portably) via getaddrinfo and friends.
(Dismal enough that I ended up writing my own DNS encoding/decoding library (https://github.com/spc476/SPCDNS) with the aim of being easy to use as well as safe. It assumes nothing about the incoming packet and does extensive validation (enough that it found bugs in other DNS libraries). It also avoids dynamic memory allocations. It's used at work in two projects that get heavy use (millions of DNS queries a day) and so far has performed without incident, so well written C code is possible.)
For ex with getaddrinfo: https://gist.github.com/zwetan/0c047d36542cc6979652
I know because I did it on purpose, even if it managed code written in AS3 you don't want to reinvent getaddrinfo, you want to reuse it "as is" as much as possible.
This is different from, for example, java bytecode which is verified and guaranteed to be memory safe.
This is not a mere technicality: an heartbleed vulnerable version of openssl compiled on top of asm.js or NaCl would still leak secret information.
edit: typo
On the other hand memory safety does not prevent bugs like Heartbleed or resent OpenSSH key leak. As the underling issue is related to broken reuse of a cached buffer, such code can be written in Java or even Rust just as well.
You are right that Heartbleed is not a memory safety issue, I should have been more careful. I just wanted an example of an exploit that doesn't need to escape the process boundary to cause damage.