Making Rust a Better Fit for Cheri and Other Platforms | Hacker News Reader

Making Rust a Better Fit for Cheri and Other Platforms

tratt.net

101 pointsltratt39 comments

The OP links Gankra's first article, but don't miss the follow-up where she actually implemented some of these ideas as proof-of-concepts in the stdlib and compiler: https://gankra.github.io/blah/tower-of-weakenings/

I think there are some problems in the article.

The main point of the referenced article by Aria Beingessner is Pointer Provenance. That this also helps out with enabling potential support for CHERI is a nice (and intended) side effect.

So yes more is needed for supporting CHERI but:

1. The non-address parts, if and how you can access them are fully CHERI specific. Any such code should (for now) not be Generic Rust but architecture specific extensions. (E.g. live in `std::arch::<somethingCHERI>`)

2. "Rust's integer type hierarchy" There is no type hierarchy in the sense there is in C/C++ as there is no sub-typing of integers (i.e. auto-conversions) in rust. Same is true for pointers so I don't think it makes sense to call `mut ()` the "root of the...". (Also if anything that would be `const ()`). Similar it's more a void-pointer (through not quite the same) then a `uintptr_t`. To quote Aria: "I don’t think Rust needs to define a moral equivalent to intptr_t [...]".

3. The 129th bit is a "secret" implementation detail. So not representing it is intended. if your code relies on somehow detecting if it's set or not to work correctly you are doing something wrong. (Through maybe for testing/asserts, still at most a CHERI specific method under std::arch.)

4. The proposed scratch design for hybrid mode fundamentally doesn't work because of pointer provenance. Pointers need to be build-in types. So in hybrid mode you now would have 4 instead of 2 pointer types. Also given that you would want to use all of them in normal code you also would have 4 instead of 2 reference types. As far as I can tell this is also a messy nightmare to use in C/C++ (Through I have to read into it first). Anyway hybrid mode is as far as I can tell irrelevant for "pure" rust applications and only becomes relevant when linking against non-capability-supporting FFI. EDIT: Provenance is often partially lost at FFI boundary anyway and you probably would want to special type the non-capability pointers and then at the boundary convert them to "normal(capability)" pointers. Only handling "normal" pointers in rust (which all have capabilities when compiled for CHERI).

> So in hybrid mode you now would have 4 instead of 2 pointer types. Also given that you would want to use all of them in normal code you also would have 4 instead of 2 reference types.

We'd need more pointer types to support other segmented architectures anyway. Real mode x86 has __near, __far and __huge pointer types.

I'm but sure people care enough to add support for segmented architectures to rust.

I mean as far as I can tell you mainly find segmentation in very very old x86 servers.

I'm sure there are still legacy systems like that running, but this is unlikely to have anything to do with rust.

I mean even most Linux distros do no longer support being run on 32 bit arch (through they still tend to support running 32 bit applications on 64 bit arch).

It would also be nice to be able to use unsigned types (like u8, u16 and u32) to index into slices and arrays up until usize (usually a u32 or u64). Using a usize often seems wasteful for indexing into small arrays and slices and casting makes the code look ugly (also dangerous because the effect of casting is checked by the developer, not the compiler). Admittedly, this would have the downside of having code that compiles on one architecture but not another so it’s probably not worth it.

Rust supports u16 pointer. Suddenly you are casting u32 to u16. And fun begins.

Exactly this!

IIRC integer literals are the blocking issue here. Bounds checking (and elision) happens anyway, but when `Index<T>` is implemented for multiple integer types, `foo[0]` becomes ambiguous.

What’s wrong with .try_into::<usize>().unwrap()?

It fails at runtime and not at compile time and it only fails if the value is out of range, so you have to test with large enough values to trigger the assertion during testing.

We want to be able to index with integer types that are smaller than or has the same size as usize, but not with integer types that are larger than usize. And we want those checks at compile time.

Unwrap isn’t an evil thing to avoid completely. The type system cannot understand all invariants, but you can, and you can make sure they are correctly implemented.

.into() will work for u8 and u16, as rust assumes >= 16b pointers.

Which totally makes sense. u8 means 256 bytes of addressable memory. You might design chips with such limited memory, but maybe high level (read non assembly) languages are not the right tool for targetting them.

It's hideous.

Code looks ugly should maybe not be a big concern in a language like Rust. It’s fundamentally ugly to begin with in many ways. And I think that’s fine because it is a language that is primarily about control.

One could argue that this is beautiful in its own way. There’s a clarity and power in being explicit while the constraints let you move on with confidence.

Function over form. I think it makes a lot of sense in Rusts' domain.

Rust really doesn't do implicit numeric conversion, and I suspect it would be hard to retrofit. But you could write a function i(...) that promoted u8, etc, to usize:

    v[i(my_u8_index)]

This could be implemented as a compile-time check, with no runtime errors. You'd have to limit which types you support as indices if you want to be portable to platforms with tiny pointers, of course.

I guess something like this might be useful inside of specialized library code that worked with lots of small vectors.

But in general, Rust heavily favors explicit over implicit in many areas. If you want lots of automatic implicit behavior, something like Scala might be a better choice?

If you were going to do that, it may be easier just to implement `Index`/`IndexMut` over `u8`/`u16`/.. for whatever type (newtype if necessary) you're indexing into. I suppose it's hard to say without context though.

You could use TiVec[1], TiSlice[1] or TiBitset[2] (shameless plug) to have datastructures indexed by the type you need, all checked at compile type. All you need is a From/Into implementation for your chosen index type.

1: https://lib.rs/crates/typed-index-collections

2: https://lib.rs/crates/tibitset

> Using a usize often seems wasteful for indexing into small arrays and slices

The opposite is often the case, as the architecture may not support offsetting a pointer by those sizes, requiring lots of casting in tight loops. Making that explicit makes the programmer aware of this.

This is one major reason that C compiler developers want all overflow to be undefined behavior, as it allows them to upgrade for loops that use smaller index types to larger ones.

That seems easy to optimize out, especially given rust's mutability guarantees. If I have a u8 and I index into an array in a loop the compiler should be free to cast it to a usize above that loop, right?

The benefit being you can store much smaller structures, plus in the case of memory unsafety you reduce attacker control.

The vast majority of strings I personally use are certainly well under 2^32 bytes, and most are probably under 2^8.

Right, better control of the index type can help you say what you mean, and help the typechecker catch more mistakes.

If I have an array representing a single-byte lookup table, it would be nice if I could say that the array's type is array-indexed-by-u8 rather than array-indexed-by-usize.

Ada did this well: it even lets you declare an array whose index is some enumerated type.

> _ it even lets you declare an array whose index is some enumerated type._

You can do this in Rust also: make a custom newtype wrapper for your array, and impl Index<CustomEnum> for the wrapped array.

Re the non-address bits of a pointer: there could be a method `shifted_by_address_width()` that just returns a same-size copy of the data (which remains unmodified) shifted over by the width of an address on the system. That way it's platform agnostic, it's still as wide as the address type, and on systems with no extra bits beyond the address, it's just zero.

> It's important to note that the cast warnings (or errors) are not enough on their own to help make code correct: additional auditing will be required. However, at the very least, this will give programmers a good idea of where to start auditing their code [4]. Some people might choose not to adapt their code, or find it too difficult to do so, but my guess is that most people will choose to do so.

I strongly doubt that most people would change their code for an ISA that barely exists.

Maybe not. But I might change my code if it makes it easier to reason about its correctness. Especially in unsafe blocks and especially if I don't have to give up anything else (like perf).

This article feels very wrong. There is so much of exposing the underlying hardware CHERI functionality to the user in everything suggested in this article. It would never fly in Rust. This post feels completely written from the perspective of C-isms rather than from the perspective of Rust.

This paragraph in particular:

> Fortunately, I believe that Aria's proposal can be adapted such that a Rust-for-CHERI can cope with both pure capability and hybrid modes. In essence, one needs explicit, separate, types for both traditional pointers and capabilities. mut () suffices for the former and a wrapper of some sort, e.g. Cap<mut ()> for the latter. Conceptually this means that mut () is no longer the root of the integer/pointer type system, because one cannot convert Cap<mut ()> to mut () without losing information (the capability's extra bits). My gut feeling is that in practice, most code can treat mut () as the root of the integer/pointer type system, and only code which really cares about capabilities need know of Cap's existence. An additional nice property is that one will be able to write Rust code in a way that can be agnostic about pure capability mode (where size_of::<mut ()>() == size_of::<Cap<mut ()>>()) and hybrid mode (where size_of::<mut ()>() < size_of::<Cap<mut ()>>()).

Capabilities should not be visible to users of the Rust language what-so-ever and should only exist in the compiler, and nowhere else.

The more correct solution is to just forbid "hybrid" modes, which seem only useful for C projects where there is a lot of legacy code. Something Rust doesn't have.

The author also admits as such in their footnote on hybrid mode having "many uses":

> In the context of Rust, it's also worth asking whether it's worth making all pointers double width (which, though perhaps small, will undoubtedly have measurable memory and performance costs). After all, most Rust code is safe, and the compiler can guarantee that pointers can't be easily misused for things like buffer overruns. Rather, I see the main utility for a language like Rust being to impose various "sub-process" like compartments, with only relatively small portions of the code needing to use capabilities explicitly.

It is perfectly acceptable to have all Rust-on-CHERI systems to have double width pointers in my opinion. This wouldn't affect software on any existing supported platforms. If you're writing new Rust code, for a CHERI system, but not using capabilities, I would wonder why you're using the platform in the first place.

I agree that you don't want non-CHERI Rust code to have to know about capabilities in any way. However, if you are using Rust for CHERI, you need to have some access to capability functions otherwise, even in pure capability mode, you can't reduce a capability's permissions. For example, if you want to write `malloc` in purecap Rust, you'll probably want to hand out capabilities that point to blocks of memory with bounds that only cover that block: you need some way to say "create a new capability which has smaller bounds than its parent capability."

As to the utility of hybrid mode, I politely disagree. For example, is it useful for safe Rust code to always pay the size penalty of capabilities when you've proven at compile-time that they can't misuse the underlying pointers? A very different example is that hybrid mode allows you to meaningfully impose sub-process like compartments (in particular with the `DDC` register, which restricts non-capability code to a subset of the virtual address space; see also the `PCC` register and friends). Personally, I think that this latter technique holds a great deal of potential.

> For example, is it useful for safe Rust code to always pay the size penalty of capabilities when you've proven at compile-time that they can't misuse the underlying pointers?

This seems like it's impossible though? How can you prove at compile time that all software that your safe Rust calls doesn't corrupt pointers? Don't you need capabilities in the Rust to ensure that if such software does something nefarious, the Rust code catches it before doing something untoward? (Not to mention the risk of compiler bugs causing something.)

If you’re passing a pointer to safe Rust code, with the capability bound encoded into something “native” to the language, then you don’t need hardware capabilities at all.

You'd still need access to the capability interface to perform that checking at the unsafe/safe boundary

Correct, but once you've done that you can strip the capability information and pass the raw address around to the safe code because the compiler runs validation.

One potential option I haven't seen mentioned is to make references (i.e. `&[mut] T`) not use capabilities, but raw pointers (`*(mut|const)`) to use capabilities. Since the compiler already guarantees that references are used correctly, at least theoretically this is best of all worlds.

Now it's possible that CHERI would make this impossible, but it's definitely an angle worth recognising.

Problem is, there's lots of unsafe code that casts *mut T to &mut T (usually after checking T is valid and whatnot). If &mut T didn't use capabilities, this kind of unsafe code would end up not taking advantage of the CHERI capability checking, which would be unfortunate.

I don't think this is actually a problem, since when casting from `&mut T` to `*mut T`, the returned pointer can only access the data (the T value) directly behind the reference.

The raw pointer would be synthesised with the capability for only the pointee of the original reference.

It's absolutely possible, because the hardware doesn't care about your compilation model: you can mix normal pointers and capabilities as you wish. A challenge is that it's easy to go from capability -> pointer, but harder to go from pointer -> capability -- where do the extra capability bits come from? CHERI C provides a default ("inherit capability bits from the DDC") but I'm not sure that's what I would choose to do.

> Capabilities should not be visible to users of the Rust language what-so-ever and should only exist in the compiler, and nowhere else.

One important use of Rust is operating systems, allocators, and JIT compilers where security is important and interacting with capabilities is expected. You’ll really want some way to represent them in the language for these usecases.

Naively I think you can probably just expose interactions with them inside existing apis like "slice::split_at_mut", since the rust abstract machine already has the concept of pointers being able to access regions of memory.

You would probably have direct access to the primitives available, but discourage using them, because using them means non portable code.

Right, the instances I'm mentioning are going to be parts of the codebase that are intended to be platform-specific.

Reply on news.ycombinator.com