Comment by jcranmer | Hacker News Reader

Comment by jcranmer | Hacker News Reader

I can definitely feel the pain of trying to work out ABI mismatch concerns. It doesn't help that it often isn't clear from the output what some of the underlying assumptions are--expected stack alignment, for example, or structs being broken up into registers or not.

It would be nice if compilers could output some sort of metadata that basically says "ah yes, here's a struct, it requires this alignment, fields are at these offsets and have these sizes, and are present in these cases" (the latter option being able to support discriminated unions) or "function call, parameters are here, here, and here." You'd think this is what DWARF itself provides, but if you play around with DWARF for a bit, you discover that it actually lacks a lot of the low-level ABI details you want to uncover; instead, it's more of a format that's meant to be generic enough to convey to the debugger the AST of the source program along with some hint of how to map the binary code to that AST--you can't really write a language-agnostic DWARF-based debugger.

Some Common Lisp FFIs have opted to coax this information out of the compiler. https://github.com/rpav/c2ffi is a C++ tool that links to libclang-cpp and literally outputs JSON with sizes and alignments. (It is then used by https://github.com/rpav/cl-autowrap to autogenerate a Lisp wrapper.) The older CFFI Groveller [1] works by generating C code which is compiled by the system C compiler (e.g. GCC or Clang) and, when executed, prints Lisp code that contains resolved values of constants, sizes, alignments, etc.

[1] https://cffi.common-lisp.dev/manual/html_node/The-Groveller....

Very lisp. Basically reprogram itself. Unfortunately this is not applicable to maintained code like c, rust … etc?

In fact, in a way C and Rust do the same thing!

When you run ./configure or cmake for a C program, it often prints something like "configure: checking size of long long" or "-- Check size of long long". This is done by generating, compiling and running a short C program that prints sizeof long long. The result goes into an autogenerated config.h.

In Rust the first example of build.rs usage [1] compiles and runs a C program during the build of the crate, and the next page [2] shows how to use autogenerated Rust code with include! macro.

Lisp is more similar to C or Rust than you might think. Code generation typically happens while the library or program source code is being loaded, and it is orchestrated by a declaration in an .asd file, which is analogous to meson.build, but looks more like Cargo.toml, e.g. [3]

[1] https://doc.rust-lang.org/cargo/reference/build-scripts.html [2] https://doc.rust-lang.org/cargo/reference/build-script-examp... [3] https://github.com/rpav/cl-freetype2/blob/b7871aed0c5244fc3b...

What you are asking for sounds quite a bit like what rustc does.

Rustc (outside `extern "c"`) offers no guarantees on the ordering of the fields, however, it guarantees that every instance of struct A will have the same ordering during that particular compilation. This allows rustc to compile external crates (as long as no monomorphization is needed) in a consistent manner across all crates that depend on that.

Most of the ABI issues arise when you start to mix and match shared libaries produced by different compilers, or even the libraries produced by the different versions of the same compiler.

Rust has none of that, nor does support dynamic linking, so I fail to understand what is it that rustc can offer in that solution space. There is none.

Rust works around the issue by not allowing all the useful things that get you there. There are other useful things like sharing pointers across threads that rust will not let you do - for both better and worse. (better in that you avoid a lot of problems for something you rarely need - worse for those few cases where you actually need to do those and cannot)

You can share pointers across threads in Rust, it's just `unsafe`.

Reply on news.ycombinator.com