Hacker News Reader

Comment by shmerl | Hacker News Reader

How does it manage C++ bindings? Rust struggled with that, and it has a lot of issues because of compiler specific name decorations and etc.

https://en.wikipedia.org/wiki/Name_mangling#How_different_co...

View on HN

The difficulty in importing C++ libraries from other languages don't really represent a Rust issue, but rather a C++ issue, of course.

As to the very interesting question of how it's handled here, it looks like part of the project (the "qtdrv" folder) is a shared library written in C++ that includes Qt, wraps the desired functionality, and exports it as C-style symbols (with extern "C"). This is a pretty standard way of circumventing the name mangling problem.

What I find interesting is that everything appears to be wrapped into a single exported function "qtdrv". Might be a wrapping technique that I'm not aware of, or maybe I'm completely misreading the source. I'm actually quite interested in knowing how the wrapping code was generated, having projects where the same kind of C++ wrapping is required.

I guess success of such approach depends on how well C++ logic can be wrapped in C. It's not always straightforward or even possible.

I haven't looked into the source, but Qt use a "MOC" pre-compilation system to generate introspection data. Once you have access to that, you don't have to mess with C++ symbols anymore. Same for the objects properties, you can get/set them by name.

In principle, yes, but a) access via MOC is certainly slower than compiling a direct function call into a single `call` opcode, and b) MOC only covers signals, slots, and Q_INVOKABLE methods. For QObjects, that means that the remaining methods are not accessible. And methods for non-QObject-derived classes are also inaccessible.

Rust has #[no_mangle] and ffi types. I've done some FFI in Rust, and I've found it to be pretty pleasant. What are the "lot of issues" you're talking about?

Rust has good C FFI. The Book (https://doc.rust-lang.org/book/ffi.html) itself says that Rust is not able to call directly into a C++ library.

Rust can call directly into a C++ library as long as it uses the C ABI: there aren't bindings to (a compiler-specific implementation of) the C++ type system. Go is just the same. This isn't really a language restriction, more a result of how rich the C type system is and how compiler-specific it is. The only languages that have direct C++ interop are those that directly use a C++ compiler (like Objective-C++).

It appears that GoQt has this giant pile of wrappers around C++ types, casting pointers back and forth:

https://raw.githubusercontent.com/visualfc/goqt/master/qtdrv...

I would have expected this to require an `extern "C"` declaration. I'm sort of surprised it doesn't, but maybe the common C++ ABIs match the C ones if only C types are passed to and from the function?

(Also, is this file auto-generated?)

I wonder though why C++ couldn't come up with a standard for names decoration? It would solve tons of such issues.

The answer is in the wikipedia article you linked to earlier...

    [it] would not suffice to guarantee C++ compiler
    interoperability and it might even create a false
    impression that interoperability is possible and
    safe when it isn't.

It doesn't really answer the question. It's not only about compilers interoperability, but about improving situation with bindings from other languages. And making name mangling standard would surely improve it (at least somewhat).

Oh, that's a very good point: if you have two different implementations of std::string, you want them to name-mangle differently so that you don't accidentally pass e.g. a GNU std::string to a function expecting an LLVM one.

Name decoration isn't the hard part; it's types that are hard. Remember that a simple string is a std::basic_string<char, std::char_traits<char>, std::allocator<char>>; passing a string from one place to another requires either sharing the latter two, or having a way to convey pointers to what the latter two do. And that's not even considering things like, oh, throwing an exception whose class uses multiple inheritance and virtual methods.

Take a look at the Itanium C++ ABI, the de facto standard on GNU/Linux etc. for a sense of the scale of the problem: https://mentorembedded.github.io/cxx-abi/abi.html

And that doesn't even standardize the layout of the actual types in std::* themselves, just the typesystem. That just gives you enough information to correctly access someone else's implementation of std::string; you still need to have that precise implementation around at compile time. There is, however, a proposal to standardize the layout of std: https://isocpp.org/files/papers/n4028.pdf

To be fair, you have the exact same problem when trying to call into e.g. Rust from e.g. C. Rust's typesystem is complex and its libstd is involved and not stabilized, so the easiest way is, again, to route through C types instead of trying to access Rust's String type in C.

> here is, however, a proposal to standardize the layout of std

That's interesting, thanks.

It'd be an improvement if everyone used the Itanium ABI, though.

Compiler and OS vendors. The same behind undefined behavior.

Although people like to bash C++, there isn't any programming language with an ABI as part of the language standard, not even C.

The C ABI that so many people adhere to, only exists in OS written either in C or C++ (via extern "C").

By the historical accident that all commercial major OSes are mostly written in C, developers that aren't language lawyers tend to think C ABI is somehow defined in some standard.

In OSes that weren't written C, with C compilers available, like OS/400, VMS, Lillith, Lisp Machines, Oberon, the ABI being used isn't the C one.

Yes, the C ABI / calling convention is OS- and platform-specific. (Which does lead to practical problems on modern systems when you assume the ABI behaves similarly on different platforms; a good example is https://github.com/rust-lang/rust/pull/22862 , caused by 64-bit iOS treating varargs differently from OS X on x86-64 or even Linux on aarch64.)

However, within a given platform, the ABI tends to be small, clear, and well-documented. It's not documented in the C language standard, but there is documentation for it, from whoever defines the platform.

I thought I had mentioned this in another comment, but to be clear, I'm not trying to "bash C++"; plenty of other type systems and standard libraries have the same issue (including Go, Rust, Python, etc.). I'm just relaying the fact that, on the platforms where Go and Rust support FFI, they do so by implementing the C ABI, which is a stable ABI defined in the platform documentation, and the C++ ABI tends not to be well-defined or stable and is also much more complex to implement. This isn't a criticism of C++, just a fact. (And there are good things about this; for instance, C++'s type system is so much more useful than C's.)

Fair enough, my reaction was more to those that bash C++ without recognition that the issue is not C++ specific.

C++ names decoration is a problem. no_mangle refers to Rust (i.e. it's useful for calling Rust code from other languages). Rust has no control over C++ names mangling.

Okay, sorry, read your original comment backwards.

Calling into C++ is generally a big problem in any programming language though, and for most, you have to write a C wrapper.

Not a problem for Julia: https://github.com/Keno/Cxx.jl

Looks like that uses Clang/LLVM. Does it work with C++ libraries built with GCC or MSVC?

MSVC: probably not; Clang's MSVC ABI support is coming along well, but last time I checked there was no COFF support in the JIT. GCC on Itanium ABI platforms should be fine, although mixing C++ stdlib versions would be risky (primarily an issue on OS X).

Note that C++ has "extern C" for the same purpose, and many people do not seem to be aware of it.

You can perfectly write a C++ library and expose a C-safe subset. I've done so many times.

The problem arises if you need to expose objects (with virtual methods), which C obviously doesn't have as the ffi is basically restricted to plain functions and record types.

I'm wondering how they deal with Go's garbage collector, which is allowed to move objects in memory.

The Go garbage collector only concerns itself with memory allocated in Go, not with memory allocated in C-land. So, it doesn't move C objects. (C objects are typically freed using finalizers, or an explicit method call.)

Of course, moving can cause problems for Go pointers in C land, but keeping Go pointers beyond a cgo function call will be prohibited in Go 1.6:

https://tip.golang.org/doc/go1.6#cgo

Yes, the callback functions (registered in Qt, calling back into Go) will probably be a problem then. I guess they will need to handle the situation through a kind of lookup-table approach, which of course will come at a cost.

Look in the doc folder of the project. It explains the design and memory layout. Its not as difficult as rust.

I'm not sure how it can be easier or harder for some languages (to make bindings to C++). C++ is just hard to make bindings to. Even if you do it from C to C++.

Reply on news.ycombinator.com