Hacker News Reader

Comment by JoshTriplett | Hacker News Reader

Incredibly excited to see unions available in stable Rust now!

The release notes mention my RFC, but a huge thanks also to Vadim Petrochenkov for the implementation, and all the myriad RFC contributors.

View on HN

Now that unions have stabilised, is there any significant un-expressable things in C APIs left? Bitfields?

Yes, to the best of my knowledge, bitfields are the last thing in C APIs you can't express conveniently in Rust; you can handle them with careful macros, but that's annoying, just as it was for unions.

In terms of things you can express now but that could still use some work: anonymous structs and unions. You currently still have to name all the intermediate aggregates, even when the C API doesn't. See https://internals.rust-lang.org/t/pre-rfc-anonymous-struct-a... .

The tough thing about bitfields in C are their poor portability as they can get laid out differently in memory by platform and compiler. They're often avoided even in C when you're looking for a good degree of interoperation (e.g. structs used in networking or on-disk-formats, etc.).

The scary thing about bitfields is also that in the context where they make the most sense to use, writing bits to CPU registers, they are also the easiest to misuse as writing one bitfield requires both reading and writing the whole byte(minimum addressable unit), which can have unexpected effects when writing to registers. On some chips, reading registers can have different values than writing so the value you are writing the second time might cancel out what you wrote the first time, or that each write triggers something you only wanted to trigger once.

    myregister.tx_baudrate_bitfield = 1;  // Writes the whole myregister
    myregister.tx_now_bitfield = 1;   // Reads the whole myregister, then bitmasks and finally writes.

The same mistake can of course be done with manual masking also but then it's more obvious that you are reading the registers also. When the code looks like above it's very easy to not realize that it also requires a read.

Right. And we can't just say "we need bitfields", we need 100% C-compatible bitfields, for FFI.

But not very important for FFI, since bitfields rarely seem to show up in C APIs (presumably for exactly the reasons mentioned above).

A kind of similar issue appears even with enums, because it is hard to guess the size of the integer that a given enum is implemented with. This is why it some not to use C enums in public interfaces to libraries.

Gecko (and many other large C++ programs that care about memory usage of certain heap allocated types) uses bitfields often enough, and we've had to teach bindgen how to generate bitfields so that we can access them from Rust.

Were these intended as part of an external interface for Gecko? In which case I would say using bitfields (as opposed to explicit masks) was a mistake.

But yes, I can imagine that for Mozilla there is a need to mix Rust and C++ right inside an application at boundaries that are not usually considered external, in which case all kinds of non-portable C constructs are acceptable.

> A kind of similar issue appears even with enums, because it is hard to guess the size of the integer that a given enum is implemented with.

True; fortunately, Rust has a means of declaring the size of an enum, with #[repr(u8)] and similar. But you do have to figure out the size of the C enum.

I have run into issues with bitfields, and flexible array members.

I'm not familiar with flexible array members, what are they?

Having an empty array at the end of the struct which is really in-line data:

    struct foo {
      int num_elements;
      char data[0];
    }
    struct foo *d = malloc(sizeof(foo) + num);
    d->num_elements = num;
    /* Now d->data is effectively an array char[num_elements] */

IIRC that should work in Rust, Foo would be a DST.

So this is tricky.

The way custom DSTs work in Rust is super annoying at the moment, but slightly less annoying for generics.

You are free to define a custom DST that is like:

    struct Foo {
        header: u32,
        flexible: [SomeType]
    }

However, there is no way to construct this type. The most you can do is calculate field offsets and write a ton of unsafe code.

If the type was instead

    struct Foo<T: ?Sized> {
        header: u32,
        flexible: T
    }

you would be able to construct a `&Foo<[SomeType]>` from a `&Foo<[SomeType; N]>` via DST coercions.

This only works if you know the size of the array at compile time.

For a runtime sized thing you have to implement a bespoke vector-like thing. You can find an example of that code in https://github.com/servo/servo/blob/e19fefcb474ea6593a684a1c... where we have a generic "HeaderWithSlice<H, [T]>" type which can be heap allocated as a header followed by the flexible DST [T].

This could be improved. The HeaderWithSlice thing could probably be a useful crate for implementing completely-heap-allocated vectors (where the len/cap are on the heap) or shared reference counted types with flexible members. We haven't really split it out as a crate because we don't actually ever mutate it so the amount of code we need is significantly less (but it's not as useful as it could be).

So yeah, flexible array members can be implemented in Rust, but it's a lot of hacky work. It's an equivalent amount of work and unsafety to write a custom Index impl on a repr(C) type with a zero-length array at the end. The DST doesn't actually get you much here.

I've never quite understood this one.

For example-- what happens if I create an array of foos and then malloc data of each element to some arbitrary size?

You don't do that. Each one is usually allocated on its own, and you keep a pointer to each. Though you could store them contiguously if each contains a length member or they're delimited in some other way.

This page[1] gives a explanation of flexible array members. They were introduced in C99.

[1] https://en.wikipedia.org/wiki/Flexible_array_member

I was pretty surprised, how the heck are unions different from enums?

I read that they're essentially the same as C unions (~untagged enums), but I have no clue how `match` works in that case.

A pattern on a union just interprets it as the specified field. It doesn't try to determine which one was actually written to last.

Reply on news.ycombinator.com