Hacker News Reader

Comment by JonChesterfield | Hacker News Reader

Pointer provenance is a nice example. A block of memory cannot be read as an array of simd types sometimes and scalar types otherwise. It can't contain atomic values which are operated on using non-atomic operations during program startup before you spawn any threads.

There were proposals to let one mmap existing structures but I don't know if any landed. Usually done with reinterpret cast and hoping that rule violation doesn't break you.

Pointer provenance does make most application code faster but other times it opens a performance gap that you have to step outside of C++ to close. Compiler extensions, switching off the analysis, changing language.

View on HN

> A block of memory cannot be read as an array of simd types sometimes and scalar types otherwise.

As far as I can tell, it is currently the case that, using raw pointers, this is not actually undefined behavior (but I never entirely trust my conclusions on these matters).

"&mut T and &T follow LLVM’s scoped noalias model" [1][referring to 2 and 3] but I am fairly sure this does not currently apply to raw pointers, and "provenance is implicitly shared with all pointers transitively derived from the original pointer through operations like offset, borrowing, and pointer casts." [4]

[1] https://doc.rust-lang.org/reference/behavior-considered-unde...

[2] https://llvm.org/docs/LangRef.html#pointeraliasing

[3] "noalias" under https://llvm.org/docs/LangRef.html#parameter-attributes

[4] https://doc.rust-lang.org/core/ptr/index.html

Also excellent are

https://faultlore.com/blah/fix-rust-pointers

https://www.ralfj.de/blog/2018/07/24/pointers-and-bytes.html

https://www.ralfj.de/blog/2020/12/14/provenance.html

https://www.ralfj.de/blog/2022/04/11/provenance-exposed.html

It seems likely you'd already be familiar with these; I'm just putting them out there for anyone interested.

LLVM can represent various aliasing relationships, modulo some risk of C++ inspired bugs in some passes. They might all be stamped out now. I remember a bug report about one that was open for many years.

I'm happy to hear rust can (probably) represent the same relationships LLVM can. C++ cannot, at least as of about two years ago when I last looked through the corresponding papers. All it can do is different types do not alias, where atomic_int and int are different types.

No, LLVM definitely still has big problems. https://github.com/llvm/llvm-project/issues/45725 is an example, the symptom in Rust is that you can write what is in effect a pointer comparison in which LLVM ends up claiming that two things are different, although they are also identical...

Use of mmap itself is undefined in the language.

Posix provides a definition that programs rely on, instead. Implementers are allowed to define literally anything the union of all standards leaves undefined.

Mmap itself is alright. You've got a void* from somewhere, that's OK. You can placement new into it to make objects.

What isn't allowed is casting it to a hashtable type and then using it as such. Because there is no hashtable instance anywhere, and specifically not there, so you've violated the pointer aliasing rules.

The obvious fix is to guarantee that placement new doesn't change the bytes, perhaps only for trivially copyable types or similar constraint. I didn't see the proposals in that direction land but also didn't see them fail, so maybe the newer standard permits it.

As I understand it, that's precisely what std::start_lifetime_as<T>() does: it effectively performs a placement new to create a T object, except that it retains the existing bytes at the address. It only works with implicit-lifetime types (i.e., scalars, or classes with a trivial constructor), though, so it probably wouldn't work with your hash table example, except perhaps for an inline hash table.

Superb! Looking through https://en.cppreference.com/w/cpp/memory/start_lifetime_as, this appears to be the right thing. It also has volatile overloads (which it looks like placement new still does not). This doesn't appear to be implemented in libc++ yet but that seems fixable, it'll go down the same object construction logic placement new does. Thank you for the reference, that'll fix some ugly edge cases in one of my libraries.

To call mmap, you are calling a function that is not in the collection of translation units that makes the program. Libraries are beyond the Standard. Include files not listed in the Standard, likewise. So you rely on Posix, there.

For objects got by casting void* to a known type, you rely on the compiler being unable to prove that the objects didn't exist already, somewhere in the program. Pray the compiler doesn't get smart enough to notice no constructor for that type is linked, meaning you that you couldn't have made that object.

Can't you also just say that mmap() is a magical function that you don't know what it does?

For all the compiler knows, mmap() could just be a:

  static Hash_Table h; return (void *)&h;

And make that the rule for all externally defined functions.

Indeed. That's the way to reason about correctness of opaque functions.

Reply on news.ycombinator.com