Hacker News Reader

Comment by j-pb | Hacker News Reader

Assembly in the macro-assembler sense. Doesn't matter that it's obscure and not a direct mapping to any specific ISA, there is still relatively straightforward mapping between syntactic constructs and something von neumann-ish.

Rust simply lacks that mapping, yet.

View on HN

Rust has that mapping in the same sense that C does: there exists a fairly simple way to implement every language construct (except maybe for dynamic method invocations) in assembly.

However, Rust and C alike have another whole dimension to their semantics, that of Undefined Behavior, which is not reflected in the assembly, and which needs to be taken into account for unsafe code authors (in Rust) / by all programmers (in C). See for example https://www.ralfj.de/blog/2019/07/14/uninit.html for what goes wrong when you think of C as just a macro assembler.

Rusts entire trait model is a pretty big deal. The main issue is with memory semantics however, stuff like tagged 64bit pointers are UB land pretty much immediately. MiniRust solves this by providing an explicit memory interface trait.

Yeah okay, traits are super complicated and C doesn't have anything like that, I admit. It's not really part of how to map instructions to the machine, it's more part of how to figure out which function is even being called at a particular call site.

> The main issue is with memory semantics however, stuff like tagged 64bit pointers are UB land pretty much immediately. MiniRust solves this by providing an explicit memory interface trait.

But this is no different from C! The way C programs interact with memory is defined in very similar terms to MiniRust's memory interface trait, using "objects" with some rules for which pointers are allowed to interact with which objects.

In both C and Rust, if you want to do pointer tagging or bit packing, you need to consider the language's rules carefully- this does not mean you cannot do it in either case, only that you are stepping close to the boundary of what is supported.

Yes, hence my delight at the creation of MiniRust.

To be clear, MiniRust doesn't change any of the rules Rust has, it makes them precise. 64bit tagged pointers are possible in Rust, you just have to do it right, and ideally one day MiniRust will spell out precisely what "right" means.

(Maybe that's what you meant, it wasn't entirely clear.)

Yeah that's what I meant.

I'm completely fine with difficult semantics in weird corners of the language. I just want to know them tho :D.

The distintion between Rust and C looks interesting. Are there any formal (or formal-ish) grounds for saying that only authors of unsafe code need to be aware of Undefined Behaviour in Rust? With MiniRust, would it be plausible that some sort of "black-box abstraction" theorem could be proven, in a way that makes "safe" code depending on "unsafe" code insensitive to undefined behaviours?

> Are there any formal (or formal-ish) grounds for saying that only authors of unsafe code need to be aware of Undefined Behaviour in Rust?

Yes. It's called type safety / type soundness: you cannot cause UB in safe code.

I literally did a PhD on that topic: https://research.ralfj.de/phd/thesis-screen.pdf

If we understand "straightforward" in the sense that small changes in the input lead to predictable and small changes in the output, then I can't see how this is reasonably true for pretty much any optimizing compiler.

Just today we had this nice example on the front page where initializing a variable differently completely changed the codegen for large parts of the program: https://jpieper.com/2022/08/05/debugging-bare-metal-stm32-fr...

Rust screams UB much quicker when it comes to memory semantics than any other language that I know. Rusts lower bounds of code mangling are also much higher than in other languages.

I don't expect to have a good mental model of an optimising compiler in "gotta go fast"-mode, but a mental model of what the memory layout looks like is pretty darn relevant when designing or working with ABIs and binary data representations.

> Rust screams UB much quicker when it comes to memory semantics than any other language that I know.

If you only use raw pointers, Rust has significantly less UB that C does.

But if you use references, then yeah we have those aliasing rules and they can be quite restrictive.

My brain isn't cached into the issue anymore, because I've since solved the same problem with Zig, but about a year ago there was no way to have a reference counted tagged pointer that points to a boxed DST with the minimum number of bytes. Every path lead through UB or "not yet defined" somewhere.

You wrote a PHD on it so you might have a better idea of the issues that past me ran into, than what present me can still recall ;)

It might still be tricky or even impossible to do that, that sounds very specific. If it is truly impossible we in the UCG would love to hear that. :) We don't want to rule out any legitimate use-case without reasonable alternatives (and yes those are deliberately fuzzy terms), but we do want to guarantee some baseline of optimizations that almost all code can benefit from, and it is okay for us if niche code then becomes harder to write. If we accidentally make it impossible to write though, that would be an issue we'd love to know about.

I have it on my todo list to revisit the Rust implementation at some point. Now that const generics have landed at least some parts of the code should be easier to write.

I'll throw you folks an issue over the fence, should I run into the same problems again, pinky swear ;)

Whats the current state of the art for unsafe code? The guidelines?(https://rust-lang.github.io/unsafe-code-guidelines/)

I found the Rustonomicon, despite it's mythological status, to be quite thin. ^^'

UCG and their Zulip are a good place for thorny "is this UB" questions, yeah. We're thin on documentation indeed. First need to figure out all the rules, then find enough people to write all the educational material to teach them to everyone else. ;)

> I don't expect to have a good mental model of an optimising compiler in "gotta go fast"-mode, but a mental model of what the memory layout looks like is pretty darn relevant when designing or working with ABIs and binary data representations.

I'm not sure where you got the idea otherwise, but Rust supports the same memory layouts and ABIs as C on a given platform.

You don't get a specific layout by default because that allows for the implementation to improve over time, but that's not really relevant when working with binary data representations, where you simply specify which one to use.

It's also worth noting that (at least for C), the way the standard is worded an optimizing compiler and a non-optimizing compiler can generate exactly the same output in every situation. NONE of the unpredictability actually depends on optimization, it's just that optimization can (sometimes, if you're lucky) expose it. If you're not lucky, the code gen changes based on something harder to control than a single compiler option.

This is still wrong though, since UB doesn't exist at an assembly level, and yet C has it. Therefore there is no clear mapping from C to assembly (even considering hypothetical ISAs).

> UB doesn't exist at an assembly level

cough https://riscv.org/wp-content/uploads/2018/05/10.45-clifford-...

There are plenty of undefined things in ISAs, writing to some internal ARM registries also springs to mind, but that's a bit of a red herring. Just because code with undefined behaviour doesn't map to some abstract machine model nicely, doesn't imply that code that doesn't invoke UB can't have a simple mapping to some abstract machine semantics.

Yes, things like illegal opcodes, or other things the hardware _does_ but isn't officially part of the spec... those things are almost as old as computers. Certainly as old as home computers.

The VSP hack on Commodore 64 comes to mind.

> Doesn't matter that it's obscure and not a direct mapping to any specific ISA, there is still relatively straightforward mapping between syntactic constructs and something von neumann-ish.

No there isn't. "Something von neumann-ish" would have behaviour for all inputs - maybe not desirable behaviour, maybe even different behaviour on different processor revisions, but it would have behaviour. The C abstract machine doesn't.

> Assembly in the macro-assembler sense. Doesn't matter that it's obscure and not a direct mapping to any specific ISA, there is still relatively straightforward mapping between syntactic constructs and something von neumann-ish.

That applies to pretty much all imperative languages that compile directly to assembly/machine code – including Rust.

Reply on news.ycombinator.com