Hacker News Reader

The idea of the `unsafe` keyword in Rust isn't that a single use of it anywhere in the dependency chain must make the entire code base suspect. Instead, the point is that encapsulation can be used to create primitives that rely on `unsafe` internally but whose interface doesn't provide any way to violate memory safety; downstream consumers of the interface shouldn't need to worry themselves about upholding memory safety.

Here's a contrived example, imagine a library with the following public function:

    pub fn index_or_zero_i_guess_lol_idk(array: Vec<u8>, index: usize) -> u8 {
        if index < array.len() {
            // SAFETY: array index must be in bounds
            unsafe { *array.get_unchecked(index) }
        } else {
            0 // yolo
        }
    }

Does a downstream consumer of this library need to do anything to ensure memory safety of their own code? No, because the unsafe `get_unchecked` function has exactly one caller-upheld safety invariant, that the array index is in bounds (documented at https://doc.rust-lang.org/std/primitive.slice.html#safety ), and because this code makes sure to uphold the invariant, the buck stops here. A downstream consumer need not use `unsafe` at all, and thus not give up the guard rails. And even in a single codebase, `unsafe` only "infects" the module that it's in, not the whole codebase, since a module is the unit of encapsulation in Rust. If you make your modules containing `unsafe` as small as possible, then you reduce amount of code that needs audited for safety.

And that's the whole point of `unsafe` in Rust: to reduce the scope of code that can potentially be memory-unsafe. It firmly places responsibility on users of `unsafe` to ensure that they uphold the necessary safety invariants; there's never any question as to which code is responsible for memory unsafety, as it can always be traced back to a single use of the `unsafe` keyword somewhere that has failed to uphold a safety invariant. It doesn't outright eliminate bugs--humans are fallible--but it makes it obvious when they might happen, it makes it easier to have confidence that they won't happen, and when they do happen it's easier to figure out why.

Your contrived example makes sense, I guess I'm just confused about how one would have faith that code using `unsafe` is actually safe? As in, to me (an avid non-Rust-user), seeing unsafe would mean either A.) I have to audit this codebase to make sure there wasn't any legitimate unsafe behavior, or B.) I just have to totally skip this dependency.

If the argument is "You can use unsafe but do things safely", isn't that the same argument as C? "Just have faith that they did it right"?

The difference is in the quantity of code you have to audit. In a given Rust project perhaps 0.1% of the lines of code may be unsafe, which while technically non-zero is widly less effort to audit than the entire codebase in the case of something like C

In addition to what the sibling comment says, you indeed often can find dependencies with no `unsafe` keyword usage whatsoever, and libraries that do so will tout this as a feature. Fortunately for your purposes the broader Rust population (including library authors) is mostly composed of people who are more-or-less skeptical of wanton `unsafe` usage in the first place (as should be self-evident, as otherwise they'd probably just be using C or C++!).

Yeah I suppose I would've been inclined to just throw out any deps that are built atop unsafe. It seems to kind of totally ruin the intended purpose of Rust as a language (to me, who is uninvolved).

"unsafe" doesn't always mean unsafe.

For example, you wish to implement a Vector in Rust.

You generally don't want to allocate that Vector over and over, so you request more memory than you need.

That "extra" memory is uninitialized which is a big no-no in Rust. You might be able to initialize that memory, but that will have performance implications that C/C++ programmers would just laugh at.

However, your "invariant" is that you never access that memory until you have a real element that you put there. That's perfectly fine.

Nevertheless, accessing that uninitialized area will always be "unsafe" from the Rust compiler point of view.

There has to be some unsafe, or else you can't have the parts of the standard library that interact with the OS. I'd at a minimum also include projects with experienced authors and similar needs such as tokio.

That's perfectly valid, and you would likely benefit from the cargo-geiger tool mentioned above (or just plain-old grep for "unsafe").

This does assume that all `unsafe` usages are wrapped in correctly `safe` guards though, doesn't it?

You can wrap `unsafe` functionality without checking and thus have memory issues, or if the unsafe wrapper hasn't correctly validated all possible entries that can cause memory corruption. Then any consumer of this dependency has a possible security issue with the right inputs to the consumer, which can trigger passing an exploit down to the unsafe portion.

Yes. If your safe code blows up but everything else seems fine, the fault is one of (starting with least likely):

* Your hardware is broken. Did you spill the entire glass of wine on that laptop?

* The Rust compiler has a bug. Or, equivalently, LLVM has a bug and rustc depended on LLVM to be correct. Or, your OS kernel has a bug. These are all unlikely, but they do happen.

* A safe wrapper in a popular library you depend on is not fulfilling its contract and you noticed before anybody else.

* A safe wrapper in your code, or in some lesser known crate you depend on, is not fulfilling its contract. If you just fired Jim for writing LGTM without reading his code reviews, start with any code Jim reviewed that has "unsafe" in it.

Notice that the blame definitely is not on the safe code which relied on a guarantee that wasn't fufilled. This is a human construct layered on top of what Rust's compiler is actually delivering, but it's still important to think this way.

If in hindsight the function Jim didn't review correctly should be marked unsafe because you need to check your parameter wasn't zero when calling it, do exactly that, never write...

  // Marked safe, but actually not safe to call with zero

... in the comments for the function, you're making your own life harder.

> Or, your OS kernel has a bug. These are all unlikely, but they do happen.

For fun, here's an example of the Rust stdlib itself working around such things: https://github.com/rust-lang/rust/blob/74ef0c3e404cc72c08b2d...

A downstream consumer might not need to use the keyword unsafe, but by calling such a function, they have de facto lost memory safety. It is a nice-to-have to know that only the unsafe blocks of code are able to violate memory safety, but once memory is corrupted, errors can manifest anywhere.

Rust in practice is likely not as safe an ecosystem as C#, because Rust users are more likely to use unsafe blocks for performance tricks. In my view, Rust should have a safe keyword to mark functions that are transitively not unsafe, much like the pure keyword in some languages marks transitive functional purity.

> A downstream consumer might not need to use the keyword unsafe, but by calling such a function, they have de facto lost memory safety.

That's a silly definition of memory safety. You can't write a non trivial rust program that doesn't use unsafe in it's dependencies. No interior mutability, no allocation, no containers.

The point is these uses of unsafe are checked by the author to ensure they can't introduce unsound behavior through the exposed safe API.

> Rust in practice is likely not as safe an ecosystem as C#, because Rust users are more likely to use unsafe blocks for performance tricks.

Sort of agree with you here. Unsafe code is rare in C#. But C# has race conditions so some kinds of bugs are more common. But C# isn't as suited for systems programming type of problems or really high performance code.

It is an honest definition of memory safety. The fact that you can not write a non trivial Rust program without unsafe dependencies is not an argument against that definition.

Most languages considered memory safe have some sort virtual machine model where non trivial programs can be written without any use of unsafe. The virtual machine implementation itself may be unsafe of course, but that is a pretty solid boundary. In Rust, that boundary is easily crossed by anyone, for any reason, to the point where it sips into 50% of dependencies, as this paper shows.

I am not saying that C# evades this issue in principle, obviously it has the same functionality, it is just not used as liberally. Concurrency is orthogonal.

> Most languages considered memory safe have some sort virtual machine model where non trivial programs can be written without any use of unsafe.

With the obvious exception of JS in the browser, I don’t believe this is true: Python, Ruby, Java, C# all have widely used unsafe native escape hatches that are regularly found in dependency trees. All you have to do is misuse these interfaces, which is significantly easier than in Rust.

Read carefully: The premise is that you can't write a non-trivial program in Rust without using unsafe.

It's true that most language implementations have an escape hatch in the form of a foreign function interface and/or they have native extension modules, nevertheless non-trivial programs can be written without using them. Conversely, using such facilities is often a reason why a program may not be portable between multiple language implementations.

No, that's not my premise. You can't write a non trivial rust program that doesn't use unsafe in it's dependencies, often the standard library.

But this is true of every other programming language I'm aware of.

So if you want to define memory safety to be something never achieved in practice, fine, but that's not a useful definition in my books.

> No, that's not my premise. You can't write a non trivial rust program that doesn't use unsafe in it's dependencies, often the standard library.

Same difference, because the use of unsafe is transitive. There is no distinction between "unsafe-but-part-of-the-standard-library" and "unsafe".

> But this is true of every other programming language I'm aware of. So if you want to define memory safety to be something never achieved in practice, fine, but that's not a useful definition in my books.

I don't. I already made that distinction further up in the thread, you can read that and reply there if you have any objections.

So nothing is memory safe by your definition then.

Languages like C# are more memory safe than Rust because the uses of unsafe code are fewer and more concentrated in the runtime where bugs are more easily caught. I think that's a fair comparison.

> So nothing is memory safe by your definition then.

No. I will now apply maximum charity and explain the distinction once again: Let's take Java. It runs on a VM. The VM has a memory model. There is no way to write memory-unsafe code while staying within that model. Not only can you write non-trivial programs against that model, the vast majority of Java code is written against it and is thus memory safe. That's a fair definition of memory safety.

Of course, in practice, you will likely use an implementation of that language that uses unsafe memory access. You will likely use an implementation that has escape hatches to call into native code - unsafely. That's a reasonable boundary to have while still calling the language memory-safe. Unsafe in C# was a late addition that can usually be disabled without much peril.

> Rust in practice is likely not as safe an ecosystem as C#, because Rust users are more likely to use unsafe blocks for performance tricks.

But this suffers from selection bias: people don't tend to write the same sort of high-performance libraries in C# as they do in Rust precisely because of how much harder it is to squeeze out every last drop of performance from a managed language (even taking into account C#'s own `unsafe` keyword). If Rust didn't offer `unsafe` and only privileged its own first-party constructs then it could certainly produce a more broadly safe ecosystem, but it would not produce a more useful one. And therein lies the whole point: the goal of Rust is not to produce a safe ecosystem for its own sake, but to provide a more reasonably-safe alternative to C and C++ for the sorts of things that C and C++ get used for.

That's why I would argue for a "safe" keyword. In this sense, the compiler can actually help me write safe code and encourage library authors to do the same.

Either safety really matters and performance can be given up for it, or vice versa. I do not believe that Rust is a "reasonably-safe" alternative to C, because C is already reasonably safe if written with safety in mind, but with the added bonus of being simpler. Rust is reasonably unsafe when written with performance in mind, whereas C++ is just unreasonably unsafe.

> but once memory is corrupted, errors can manifest anywhere

Yes, but those errors can only be caused by code flagged with the unsafe keyword. Which makes debugging, auditing etc. a lot more simple than in, say, C, where it could happen anywhere.

This isn't necessarily a huge help, because those errors could be triggered or manifest at any point where unsafe code is used transitively. In C, not every single line is unsafe. All that the unsafe keyword ultimately does is to force you to aggregate such lines into blocks with the word "unsafe" around them.

You can speculate that this is a significant improvement. I would argue that a disciplined use of a simple language like C would, in practice, be safer than using a complex language like Rust which has "safety" written all over it - but then liberally sprinkles unsafe code into all the gaps.

We can both agree that C++ is the worst of both worlds.

> This isn't necessarily a huge help, because those errors could be triggered or manifest at any point where unsafe code is used transitively.

Right, and that isn’t something Rust or the unsafe keyword set out to solve. Unsafe has absolutely no runtime impact and was never designed to.

Your argument that a disciplined use of C would render this useless is correct, but history has shown us a great many examples of undisciplined use of C, even in projects that start out disciplined. Rust codifies and enforces that discipline, I don’t see anything wrong with that.

You want unsafe to be something it isn’t, but I’m not convinced what you want fits with Rust’s vision. There are many, many safer languages to choose from.

> Rust codifies and enforces that discipline

I completely disagree with that. There's no discipline required to slap "unsafe" around a block of code. The discipline must be applied to the block of code itself, there is no way to enforce that. If 30% of dependencies have unsafe code in them, how would I know that the unsafe parts aren't just as sloppily written as the average C code?

Rust has yet to prove itself as a language that actually delivers on its promise of being safer than C, in practice. To that end, we must compare C written with security in mind to Rust written with security in mind. Given that Rust is a rather complex language that encourages abstraction, I'm quite hesitant to drink the cool-aid right away.

Why would memory get corrupted in this example? To put it differently: if the code didn't have the `if index < array.len()` check, clearly things would be different. How would you describe this distinction, other than "memory safety"? (Asking because it's not clear whether you're using an unusual definition of memory safety, or just didn't notice the bounds check in the example.)

I am making a general statement about unsafe functions, not the subset of unsafe functions that happen to be memory safe.

P/Invoke is not uncommon either, especially in Windows. Rust and C# offer similar guarantees.

The difference is that in Rust you can build safe (foolproof) abstractions on top unsafe code. OTOH, in C you can't write a library that returns a pointer and prevents users from misusing it (e.g. ensure they can't use it after your lib frees the data, or improperly share it between threads). In C, the risk of unsafety and misuse of APIs remains a problem for every downstream user.

For example, Rust's String type internally allocates, reallocates, and frees memory. It can have partially uninitialized buffer, and lots of other "unsafe" things.

But there's no way to cause UB when using safe Rust's String. You can't cause use-after-free. You can't cause a data race. You can't overflow the buffer. You can't even break its UTF-8 encoding. That's because String's public interface only exposes safe methods. The unsafe String internals were written and checked for correctness, and from then on they're safe to use for everyone else.

Aren't they saying though, that there could be a bug in the string unsafe code that causes UBs. You can't guarantee that (although this example is unlikely).

There could, and these kind of bugs have happened a few times already, but fortunately they are few and far between, and as the sibling comment mentioned there's some work in progress to formal prove the std lib, and because the std lib is much smaller than the amount of code that uses the stdlib, it's much less work to formally verify the stdlib than it would be to verify for instance all C++ programms using the C++ stdlib.

[1]: the most famous bug in the standard library exposing an unsound construction is the “leakpocalypse”: https://github.com/rust-lang/rust/issues/24292

That is true in general. But 1. unsafe code in the standard library is of course more battle-tested than ad-hoc unsafe code, and 2. there are efforts to formally verify Rust. http://plv.mpi-sws.org/rustbelt/ and https://internals.rust-lang.org/t/announcing-the-formal-veri...

There's a practical difference in how much of the program is high risk. If all of the risk is concentrated in a few functions, it's easier to focus on reviewing/testing/fuzzing these bits.

Rust changes the bug-spotting game from "Where's Waldo?" to "Is this a Waldo?"

"I never understood that logic."

Pragmatic techniques mystify some.

Some code is beyond the ability of a compiler to verify because the necessary information is missing. Rust acknowledges this reality and accommodates such code through unsafe. The presumption inherent in this is that unsafe is used only when deemed necessary. Yes, deemed; a decision made by some flawed soul, as opposed to a compiler. This is a pragmatic policy and it cannot be reconciled by the unreasonable.

> You can write perfectly safe code in C where virtually everything is unsafe in the sense of the unsafe keyword.

The key here: Rust SURFACE AS TYPES this information.

Is similar to this piece of code (imagine is python):

    sum(a,b)

what it actually do? WHO KNOWS. But this:

    sum(a:int,b:int):int

is informative. And the "type system" can verify its use.

So "unsafe" is part of the type system, type checks that Rust use to COMMUNICATE and CHECK the use of code. So, if I, as user see this:

   unsafe fn raw_memory(...)

I know the type system of Rust was bypassed and must TRUST the HUMAN know how correctly code what is inside.

---

The major advantage here, is that the blast area of this stuff is limited and for the rest of regular code, the type system verify the code and remove the need to manually inspect (and make tests!) for a lot of trivial (and not so much1) stuff.

Is a total game-changer, in special for a system language.

Unsafe doesn't remove all the guard rails, but it allows you to do a handful (five) of tremendously unsafe things, for which you then take all responsibility as the compiler cannot analyse them for safety.

* You can dereference raw pointers. Are they pointing at anything at all? At something that type pointer might reasonably point to? Not the compiler's job to check, if you screwed up your program now has Undefined Behaviour like C or C++

* You can access C-style unions. This one says it might be a boolean, a pointer to something, or an integer. You decided to guess it's... a boolean. If you're wrong you get Undefined Behaviour.

* You can mutate (change) static variables. Global cheese flavour is Cheddar? Let's change it to Gorgonzola, no wait Parmesan. Was anybody else using that for anything? No idea, too bad, you might introduce Undefined Behaviour if you didn't think this through.

* You can call functions Rust labelled "unsafe". These functions come with instructions about rules you must follow to use them safely. If you violate any of those instructions, all bets are off, but if you obey the instructions whoever wrote the function (which may be the Rust standard library) promises that was safe.

* You can implement special Traits labelled "unsafe" to implement. These Traits, unlike most Rust traits, have to promise they're correct. If you claim to implement Searcher, a Trait for text processing, and you get it wrong, other people's code may blow up. Whereas I have types like Funhouse<> which claim to implement the trait Eq (promising they understand how equality works) and then as a goof they utterly refuse to do so correctly, that's a safe trait, nothing blows up. My types don't work in a sensible way (Funhouses are simultaneously equal to and not equal to every other Funhouse including themselves, which is stupid), but your program doesn't have Undefined Behaviour if you use these types, they're just annoying.

But, lots of things that would cause Undefined Behaviour in some popular languages are impossible even in unsafe Rust.

Suppose I have a function that takes an array of 1024 bytes. I try to do { let z = array[1040]; } -- That won't compile, this array isn't big enough. If I write unsafe { let z = array[1040]; } it still doesn't compile, didn't I get it, this array isn't big enough.

OK, let's have the function take a parameter k, and at runtime we can set k to 1040 and try that way. If I write { let z = array[k]; } when k is 1040 the program panics. The array isn't big enough. If I write unsafe { let z = array[k]; } it still panics and the compiler even warns me that unsafe isn't doing anything for me here and I should remove it.

Similarly although Rust provides checked arithmetic (ie you can explicitly say "Tell me the answer to x + 100, or, if that would overflow, tell me it didn't work") its default integer arithmetic will panic in debug builds if you overflow and, if you missed that in debug, but it happens in production, you get overflow, which may very well not be what you wanted, but you apparently didn't even know it could happen, so, good luck with that. Either way, no Undefined Behaviour. 255_u8 + 255_u8 is a compile error, a runtime panic or at worst it's 254, but it definitely is not zero, forty-two, or unrelated program misbehaviour.

Excellent -- so your large C codebase is free of security bugs, something no software vendor has managed to accomplish. Ritchie and Thompson themselves didn't manage that feat.

Tell us how you do it, please, so the world may learn.

Oh, that’s actually simple. Enforcing the “malloc once” rule gets you like halfway there, and using custom hardened code for common stuff like string manipulation finishes the job. Things like that are why some C codebases are large in the first place - you have to reinvent your own safe space from scratch.

The funny thing is that it’s actually harder to protect from certain classes of bugs in Rust. For example, you cannot uncouple from the global allocator as easy as you can in C/C++, and if you do, you do it with “unsafe” code. That doesn’t make Rust a bad language, it’s just that you can see some of the inherent flaws only if you had written safe C code previously and then you find out that some basic techniques don’t translate.