P.S. Also, folks, don't re-use buffers without zeroing unless you absolutely need the performance and know what you're doing.
(This isn't the theoretical ivory tower kind of UB. Operating systems regularly remap a page that hasn't yet been written to.)
(E.g. imagine doing a write to an array at offset x - this is safe in Rust, so the compiler turns that into code that checks that x is within the bounds of that array, then writes at that offset. If the value of x can change, then now this code can overwrite some other variable anywhere in your program, giving you a bug that's very hard to track down)
This is not exactly an optimization though, in the sense that it will mess up even thoroughly unoptimized code (with more likelihood, due to caching optimizations being absent).
So that is to say, even the generation of basic unoptimized intermediate code for a language construct relies on assumptions like that certain quantities will not spontaneously deviate from their last stored value.
That's baked into the code generation template for the construct that someone may well have written by hand. If it is optimization, it is that coder's optimization.
The intermediate code for a checked array access, though, should be indicating that the value of the indexing expression is to be moved into a temporary register. The code which checks the value and performs the access refers to that temporary register. Only if the storage for the temporary registers (the storage to which they are translated by the back end) changes randomly would there be a problem. Like if some dynamically allocated location is used as an array index, e,g. array[foo.i] where foo is a reference to something heap allocated, the compiler cannot emit code which checks the range of foo.i, and then again refers to foo.i in the access. It has to evaluate foo.i to an abstract temporary, and refer to that. In the generated target code, that will be a machine register, or a location on the stack. If the machine register or stack are flaky, all bets are off, sure. But we have been talking about memory that is only flaky until it is written to. The temporary in question is written to!
You'd almost certainly pass it as a function parameter, prima facie in a register/on the stack, sure, and therefore in unoptimised code nothing weird would happen. But an optimising compiler might inline the function call, observe that the value doesn't escape, and then if registers are already full it might choose to access the same memory address twice (no reason to copy it onto the stack, and spilling other registers would cost more).
I don't know how likely this exact scenario is, but it's the kind of thing that can happen. Today's compilers stack dozens of optimisation passes, most of which don't know anything about what the others are doing, and all of which make basic assumptions like that the values at memory addresses aren't going to change under them (unless they're specifically marked as volatile). When one of those assumptions is broken, even compiler authors can't generally predict what the effects will be.
1. Initially map the not-yet-written page to a read-only page full of zeros (the same one for all allocations: only one exists in the whole system).
2. When a write takes place, copy-on-write clone that page to a newly allocated zero-filled-page, then allow the write to proceed.
Most functions could be inferred, but the ultimate source of basically all of these write only APIs is FFI functions, which in turn call systemcalls.
You're at least going to need a way to annotate the FFI calls and systemcalls to describe to the compiler how they access data.
If you're doing some sort of zero-copy IO, the time to clear the buffer might be non-trivial (not huge, but non-trivial). It's true that you need a large enough buffer that syscall/ffi overhead doesn't dominate, but that's not unrealistic.
It's rare that we care about this, that's true, that's why generally rust has been fine with "just zero buffers". There are definitely domains that care though.
If you can think of a different approach of how the compiler can figure out automatically what memory has become initialized by a random function call I’m all ears.
The compiler doesn’t reasonably have any of that information to understand what read is filling in at runtime because that information is encoded purely at runtime and the compiler has no reasoning mechanism even close to answering runtime data flow questions.
There’s also all sorts of complexity that has to do with the kinds of transformations that are possible as the legal information that exists at the language level is often erased before it gets to the stack/register piece and vice versa the language layer knows nothing about registers and minimal stuff about stack.
This is the same reason that the compiler fails to compile something like:
for _ in 1..10 {
let x: String = create_new_string();
eprintln(“{x}”);
}
Said another way, if the compiler could do the optimizations you’re hypothesizing, it would be equivalent to applying a mid level performance engineer to every code base it encounters.
I think it's theoretically possible, but at the cost of much longer compile times, and greater complexity in the compiler.
fn read<T, N: size_t>(&mut self, buf: &mut [MaybeUninit<T>] becomes &[T; N] after call) -> N {
… enforces the body initializes N elements out of buf
}
A more likely interface you could write today would look like:
fn read_uninit<T>(&mut self, buf: &mut [MaybeUninit<T>]) -> (&[T], &[MaybeUninit<T>]) {
… enforces the body initializes N elements out of buf
}
Problem would be: how do you express “you can only access the buffer you sent me through the read-only slice I returned, but you have to free that same buffer when you’re done calling me?
I think that can be done using a function creating a read buffer for a given input stream that
- during calls to read is ‘owned for writing’ by that stream (so, it has to borrow a capability that the creator of the buffer doesn’t have. I don’t think Rust currently supports that)
- where stream.read returns a read only slice whose lifetime is bound to that of the buffer
So, the creator of the buffer can only pass it to read to get a slice back that contains precisely the data read.
The stream can write to the entire buffer.
unsafe{ std::mem::transmute(slice) }
let slice: &mut [NonZeroU8] = ...;
let slice_uninit: &mut [MaybeUninit<NonZeroU8>] = ...;
let nonzero_uninit: &mut MaybeUninit<NonZeroU8> = &mut slice_uninit[0];
*nonzero_uninit = MaybeUninit::zeroed();
slice[0]; // Undefined behavior for sure by now.
I.e. MaybeUninit<T> allows you to write invalid bit-patterns to T, so you can't safely cast a reference to T to it (and if you do unsafely cast a reference to T to it you can't soundly write an invalid bit pattern). All current forms of safely making a MaybeUninit take ownership of the value they are declaring to be MaybeUninit for this reason.
I guess at some point we might get methods for this on types that can take on all bit patterns - if/when that's encoded as a trait.