There would be lots of problems if we tried to standardize the ABI; we would be forever locked into all sorts of decisions that would be impossible to change. For instance, we'd be unable to fix bugs in much of the standard library. This why, for example, C++ does not standardize the ABI.
> You may reorder a large struct to optimize for other properties like cache coherency
Optimizing for size is much more commonly desired than manually cache-aligning things. Optimizing for manual cache alignment would be the wrong default.
> or because you have to match a memory layout specified by other software or even hardware. Obviously one can still use 'repr(C)' for these cases.
In fact you always had to use `repr(C)` for those cases. It's just that it wasn't enforced before.
> Sometimes in C/C++ land programmers 'cheat' by adding fields to the end of a struct that's used in multiple modules, and initially only recompile the modules that need to know about the new field.
This only works if you throw type/memory safety out the window. In a language like Rust that tries to encourage memory safety, that wouldn't make sense.
> - This seems like it may violate the principle of least-surprise pretty badly. As long as we continue to use debuggers and have crash dumps, systems programmers will sometimes need to look at raw memory data and figure out what the structure is.
Rust data type layout has always been non-obvious in various ways, because of generics, zero-sized types, the various enum optimizations, etc. Not to mention that LLVM can already split apart structures on the stack in C, C++, and Rust.
> For these reasons it seems like this auto-reorder behavior might be better as an opt-in rather than opt-out behavior (which it sounds like it is.)
In C++, maybe. In Rust, it's a different story, and I'm confident we made the right decision.