old_divider = SpiRegs.CONFIG_REG.bit.CLK_DIVIDER;
SpiRegs.CONFIG_REG.bit.CLK_DIVIDER = new_divider;
old_divider = (SpiRegs.CONFIG_REG & SPI_CONFIG_CLK_DIVIDER_MASK) >> SPI_CONFIG_CLK_DIVIDER_POS;
SpiRegs.CONFIG_REG = (SpiRegs.CONFIG_REG & ~SPI_CONFIG_CLK_DIVIDER_MASK) | (new_divider << SPI_CONFIG_CLK_DIVIDER_POS);
config = SpiRegs.CONFIG_REG;
old_divider = (config & SPI_CONFIG_CLK_DIVIDER_MASK) >> SPI_CONFIG_CLK_DIVIDER_POS;
config &= ~SPI_CONFIG_CLK_DIVIDER_MASK;
config |= new_divider << SPI_CONFIG_CLK_DIVIDER_POS;
SpiRegs.CONFIG_REG = config;
They can be, if done right, but then I have to remember the names of all the helper functions and macros. :-) An IDE can auto-complete bitfield names.
>Still more work for people reading the code, though since there are a lot of hidden assumptions behind that deceptively simple "=" than with an explicit mask and shift.
Depends on the platform. IIRC on ARM a bitfield access is masking and shifting, only done by the compiler instead of me. With optimized code I often have to look at the disassembly anyway if I want to know what's really going on.
ARM supports native bitfield loading[1], extraction[2] and clearing[3].
1 - https://developer.arm.com/documentation/ddi0602/2022-03/Base...
2 - https://developer.arm.com/documentation/ddi0602/2022-03/Base...
3 - https://developer.arm.com/documentation/ddi0602/2022-03/Base...
Many of them get it right.
#define CLEARBIT(a,p) = ((a) &= ~(1 << (p)))
Otherwise you would need to define specific macros again:
#define CLEARBITLONG(a,p) = ((a) &= ~(1L << (p)))
#define CLEARBIT(a,p) = ((a) &= ~((typeof(a))1 << (p)))The standard may be flexible, but the behavior of a given compiler on a given platform will be consistent and very unlikely to change in the foreseeable future.
The only thing to be aware of here is that you shouldn't be trying to transfer data from one platform to another, loading it directly into memory, which is an insane thing to do in any case.
Testing formally-correct code can only ever find bugs in the exact compiler release and physical CPU chip you run the test on.
The bugs cited show up in various compiler releases and various chip steppings. Unless your code will only ever run on that exact machine, you have not tested it.
I don't understand this sentence. That is the purpose of the runtime check, to make sure the code generated by a buggy compiler, on a specific architecture, doesn't try to run. Bug reports can then be filed, as they should be.
If you're writing a Linux kernel module (known compiler) for a known architecture, which is where this is often used, then you have a known environment, so there's little risk, except the rare compiler bug, which is a risk that extends far beyond packed structs.
This is something that I've used, and I've seen used, often in the hardware world. I'm having difficulty sympathizing with all the fear in this comment section. Don't use it if its use can't be made rational, for you. But please don't tell me that I'm failing to get something that I am familiar with.
Perhaps there's some confusion of what a runtime check is. A runtime check is executed at runtime, which means that it will be executed after compilation with a specific compiler, on a specific architecture, in the end user system. The runtime check verifies correct struct operations at runtime, as part of the initialization, and aborts the driver load, with a nice system message, if improper packed struct behavior (among other things) is seen. This covers your concern here:
> Unless your code will only ever run on that exact machine, you have not tested it.
It literally is tested on every machine, when the driver is loaded.
This runtime check handles the cases where a customer might try to run it on some new hardware/toolchain that we don't officially support. The official supported cases only have risk of new compiler bugs, since we only support certain architectures. Hypothetical compiler bugs are a problem for all software, and shouldn't be used to drive software design, beyond making sure there is good testing, which has nothing to do with compilers.
Maybe I'm missing something.
edit: ncmncm, I can't reply to you, but since I have code that does what you say is impossible, and since my comment already responds to what you wrote, I would suggest reading all of my comments, fully, a second time.
And, what would the code do if it detected such a bug? Abort, or run other code that works OK anyway. So, run that code all the time, and you don't need to try to check.
PORTB |= (1<<4);
sbi PORTB, PINB4
PORTB = PORTB | (1<<4)l
sbi PORTB, PINB4
portb.pin4 = 1;
sbi PORTB, PINB4amd64 bugs tend to be in newer parts of the ISA. ARM provides chip makers very thorough tests, to protect their brand (though not all chip makers fix all the bugs tests find). So, many programmers will not encounter bitfield bugs. But code gets around, and tests are always less thorough than we wish.
Well, clearly that's not always the case, as shown in the article, and which is what started this comment chain.
But, I'm not sure how you reached that interpretation. I'm saying that the code that you put into the function will result in some operations, whatever they may be, inline or not. The struct access will result in some operations. The those two sets of operations may be different. Some architectures have specialized instructions for bitfield access. There's a good chance that the complier won't convert the shenanigans in your method to those specialized instructions. Maybe! It requires an understanding of your particular situation. But some people's work exist in a context where this is a reasonable choice, for them.
If you code a shift-and-mask in an inline function, instead, your odds are better, same as if you made a macro for it.
old_divider = (SpiRegs.CONFIG_REG >> SPI_CONFIG_CLK_DIVIDER_POS) & 1;
old_divider = (SpiRegs.CONFIG_REG >> 24) & 0xff;
Another "fun" issue with using numbers is that sometimes an int is 16 bits, so you have to do (13 << 24ul) instead of (13 << 24).
Not in my experience. The buggiest part was the preprocessor. You don't hear much about preprocessor bugs anymore because the C standard doesn't dare change it, and in 40 years people have finally got them working right :-/
Personally, I had to scrap and rewrite the C preprocessor 3 times to get it right.
Without bitfields the code would be absolutely filled with bit-access macros decreasing readability and screwing with IDE's indexers and static analyzers big time
Not to mention the pain it would be to refactor/reorder/change fields sizes which is relatively painless with bitfields
The main drawback of bitfields is that they work in your tests and fail in the field.
From my experience most of the bugs were just a 'normal' bugs i.e. human errors when writing and those were fixable by just figuring out what was implemented incorrectly. About 2% on a bts side were cache coherency issues because we had a multicore system without hardware coherency, so imagine, and similarily 2% were hardware issues on a modem side due to race conditions or whatever - harder to fix so workarounds. But miscompilation? x86 host tests are used to separate wheat from the chaff but the only tests anyone cares about, before commit, are on target
On modem side I remember one, maybe two if you push it, issues with miscompilation but it wasn't at all related to bitfields but the compiler was doing some stupid shit with register allocations
There are myriad places for mistakes to manifest. Caches, interrupt behavior, and sleep modes are favorite places for implementation bugs. But bitfields are a place ordinary programmers might still encounter them.
Anybody used to working with buggy one-off chip designs knows all about this. But most programmers are insulated from most bugs. The warning is for them.
I personally never code in C anymore; lately I am using C++17. Compiler optimizers are way, way more reliable than back when I used C.
I thought that was the entire purpose of the bitfield, to give you a clean (visual and operations) way to access the field:
mybitfield.fieldname = value> Multiple adjacent bit-fields are usually packed together (although this behavior is implementation-defined)
> The special unnamed bit-field of size zero can be forced to break up padding.
> int b:3; may have the range of values 0..7 or -4..3 in C
> on some platforms, bit-fields are packed left-to-right, on others right-to-left
I wouldn't touch them unless I absolutely had to, and knew I could guarantee compiler and platform.
Now you go ahead and teach GCC to use the arm UBFX instruction for those cases. It DOES use it for actual bitfields. shift + mask = 2-3 instructions (immediate load may be needed). UBFX is one.
Compiler implementors don't like to guess, but don't get a choice. If the instruction provided doesn't match the Standard, which do they implement? Both choices are wrong.
If your code only ever runs on the physical machines where you test, or only on very mainstream chip designs, then fine.
You have been warned. Now it is on you.
The best "bitpacking" I have ever dealt with is the "Erlang Bit Syntax". I really wish more languages would adopt it.
See: https://www.erlang.org/doc/programming_examples/bit_syntax.h...
1 - https://stackoverflow.com/questions/58493193/ada-how-to-expl...
I write command line tools in Ada, and some other things, I've found this bit layout very useful when reading/writing binary file formats, or when binding to C and matching C struct layout.
One time, I was working with an older embedded PPC architecture on a driver to talk to an FPGA that was attached to a 32 bit local bus. The problem is that it didn’t support unaligned accesses. The lower two address bits simply weren’t hooked up, so any access to byte addresses that weren’t a multiple of 4 would just behave as if you masked the two lower address bits off.
There was a structure that used bitfields to access bits in a register on that FPGA. It worked fine til I updated GCC, then it stopped working.
It turns out that the newer version of GCC would do a single byte unaligned read if you were accessing, say, bits 8-15 in a 32 bit bitfield, whereas the older GCC would read the full 32 bit word and shift/mask as needed.
It turns out you can force the older behavior with -fstrict-volatile-bitfields.
Took a minute to figure out, but I learned my lesson. That said, I don’t think you should really be doing IO that way anyway. I typically use IO accessor methods, sometimes with raw addresses, sometimes with struct overlays and taking the address of the member.
... but that's what a bit field is?
Not always. Switch your example to AARCH64 and check out the BFI instruction.
void bar(struct y *s, unsigned int foo) {
s->c = (s->c & 0xf0) | foo;
}A bit field (in C/C++) is a weird object type that can only exist in a structure or union type, which kind of acts like an underlying regular integral type except for those situations where it does not.
For an example of why compilers might have issues compiling bit fields properly (although this requires C++, since C's ternary operator works on rvalues, not lvalues):
struct A { int x: 3; int y: 5 } a;
(choice ? a.x : a.y) = val;
If you must use bit fields, make them unsigned. Bugs love to hide under signed bit fields.
Unsigned bitfields are a nice way to get modular arithmetic with n bits without syntactic clutter.
Appear to be. Are, when all the stars align. Are not in fact, often enough that you are issued a red warning you may ignore if you are insulated from all consequences.
The first is lvalues. In compiler jargon, an lvalue is a kind of object that can have a value stored to it. And you can usually represent it as the address of some memory location [1]. Of course, bitfields break this representation: you need to know what the bit offset and bit size of the field you're storing is (as well as the signedness).
The next level of complexity is the conditional operator. This means that, when conditional operators yield lvalues [2], you now end up in a situation where the lvalue now has a conditional bit offset and bit size within the address. Or maybe one leg of the expression returns a bit-field and the other leg returns a regular int lvalue. Imagine how complex your datastructure needs to be to represent an lvalue during this code generation phase.
[1] Not all lvalues need to have memory locations. But if you're writing a C compiler, it's an easy first approximation to give every variable, even those marked register, some memory location and rely on an optimization pass to convert stack memory locations into register locations, rather than keeping track of this information when the frontend does code generation.
[2] As mentioned elsewhere, conditional operators in C do not yield lvalues. But conditional operators in C++ do.
It also doesn’t seem like something that would come up very often. I can’t think of the last time I conditionally stored to one of two struct fields, if I ever have.
The much more normal case would be:
val = choice ? a.x : a.y;
The broader point is that bitfields are actually weird little objects that look a lot like regular objects in many, but not all, contexts. And it's very easy from a language design or implementation perspective to forget to account for the possibility that you're dealing with a weird little object. This leads to underspecified language specifications and compilers that crash if you do something weird (but legal) such as virtually inherit from a struct containing a bitfield as its last member.
[1] So challenging, in fact, that Clang gives an error message "cannot compile this conditional operator yet". It does work in g++, icx, and MSVC though.
Of course, if you go reach for C's standard "fun with lvalue" operations, you can get some crazy nonsense. What machine code should you generate here [1]:
struct A { int x : 5; volatile _Atomic int y: 3; } a;
a.y++;
[1] Whether or not you can make a bitfield _Atomic in C is implementation-defined, so it's possible that someone writes a C implementation where this is legal. I will note that, in a rare display of sanity, all C compilers I can test do in fact sensibly reject _Atomic bitfields, but for the purposes of argument, assume that someone has one where it's permitted, since it is allowable by the standard.
[2] Or programmer bugs blamed on the compiler. This is the intersection of two areas that are notorious for underspecification to begin with, and combined with the general tendency of programmers to expect C compilers to be a thin veneer over assembly, makes it awfully difficult to figure out which behavior is language-intended.
The underlying problem has to do with whether the IR has first-class concept of arbitrary lvalue or whether the frontend has to convert lvalues that get passed around to some pointer-like thing.
It might look irrelevant for discussion of low-level AOT compilers, but it is also interesting to compare how this is implemented in dynamic/“scripting” runtimes and how the choice of underlying implementation of the concept of “lvalue”/“place” influences the user visible language. Somewhat notably first draft of Common Lisp had something akin to first-class lvalues and the final standard replaced all that with significantly simpler mechanism that purely relies on macros.
He is describing a trivial difference between C and C++ that is not the problem you are being warned about.
choice ? ((a.x = val),a.x) : ((a.y = val),a.y);
For example, in the D compiler the `while` loop doesn't survive the semantic pass, as it gets rewritten into a `for` loop. The `for` loop then gets rewritten into `if` and `goto` statements. The code generator only needs to learn about `if` and `goto`.
You throw away structured control flow in favour of unstructured control flow?
I would have thought it'd be the other way around and you'd be trying to recover nice clean structured control flow from raw concepts like goto.
So, in short, structured control flow--or at least a sufficient subset of such structured control flow--can be easily recovered from low-level information, and you're generally not going to lose much information going down to that level. LLVM even has a way to attach metadata to loops despite not having any dedicated loop construct.
[1] You only need two of these concepts: the third falls out from the definition of the other two.
That's right. It sounds counter-intuitive, but it works great. You wind up with a collection of blocks of code connected by edges. Then, you can use graph theory to work magic on them in a general, correct way. Data flow analysis is based on this.
One thing the graph math does is enable the reconstruction of loops out of the blocks and edges - so you can write loops any way you please, and the compiler will figure it all out and apply general algorithms to it (like loop rotation, loop unrolling, etc.).
But hey, if Graal works for you, great!
The compiler's job is to emit instructions that do precisely the things the code says to do, as efficiently as possible. There is no need for chips to understand the purpose of the code; they just need to do what it says. They don't get confused.
Which requires a high-level understanding of the program and its control flow.
The `for` loop construct does not offer the compiler any more understanding of the code than one made from `goto`s. They contain the same information, and are interchangeable. The latter, however, is more amenable to applying mathematical algorithms to. The former is more amenable to human understanding.
With structured control-flow I can do things like reason about the level of nesting of the loop that I'm in. I can peel a loop iteration by literally saying 'take this loop here - copy the body of it out'.
For context here's the kind of structure we use https://chrisseaton.com/truffleruby/basic-graal-graphs/#loop....
struct foo {
char a : 4;
char b : 4;
};
x = foo.a
x = (foo & FOO_MASK_A) >> FOO_SHIFT_A
foo.a = x
foo = (foo &~ FOO_MASK_A) | ((a << FOO_SHIFT_A) & FOO_MASK_A)The more frequent perceived use for bit-fields (in the situation where they actually work) is to pack into a serialized data format, such that memory or a data stream can be accessed elsewhere. In that case, "the compiler can do whatever it wants with your data packing" is pretty useless, since your "elsewhere" might have a different compiler that does a totally different thing.
Ladies and gentlemen, this thought is why we now consider 8GB of ram to be a "weak device".
No, no no no no, 1000 times no. Every situation is a low ram situation. Every!
Edit: I know it's hard to read a whole sentence at once, but I made that same point directly up there too.
And as for the second part: anything that writes sizeof(struct foo) bytes of struct foo is inherently non-portable. If you portably want to (de)serialize something you want to write the thing explicitly, very often the compiler will optimize it to more direct implementation. (And well, this is only portable to platforms where CHAR_BITS == 8)
Anything that affects the actual instructions executed on the actual chip they're executed on may make what works here not work there.
Optimization that does not affect instructions is no optimization at all. Bitfields are an extremely fragile part of implementations. Trust it at your own risk.
You know where the bits are within a single word. But if you have a struct with multiple fields, it’s not safe to rely on the exact memory layout even if it doesn’t have any bitfields.
If you need to represent a very specific memory layout, it’s not just bitfields you need to avoid, it’s structs in general.
Conversely, if you don’t need to guarantee a specific layout, bitfields are fine to use, and could be a useful optimisation hint for the compiler.
Say I have a window manager, and I want to attach a bunch of boolean flags to each window object (isVisible, isMaximized, etc). I don’t need to serialize them to disk. It’s highly preferable that they should be efficiently bit-packed, but not strictly essential.
The conservative way to implement that would be bit-shifts and masking (either manually or via a macro). But implementing it with bitfields would be a lot easier and less error-prone, and would work just as well. What problems do you see with the bitfield approach?
If it works on your particular compiler release, on your particular CPU chip stepping, that tells you nothing about the next compiler over and the next chip over.
amd64 and arm64, compiled with gcc or clang, you are unlikely to run into these problems. But code tends to get around.
If so, I think that’s overly paranoid. The examples that are being given here are baroque usage that would immediately stand out in a code review - memory-mapped registers, conditional lvalues, volatile and atomic fields.
The point I wanted to make is that simple straightforward usage of bitfields, like the example I gave, works fine on any platform you’re likely to encounter.
There’s plenty of widely-used code out there that uses bitfields. I just did a code search to check that (the particular example I was thinking of comes from iOS) and found some in Clang - funnily enough, in its representation of lvalues!
64-bit Linux distros and the BSDs follow the convention once set by the "C ABI for Itanium".
In that, bitfields are grouped in declaration order into container words of the same width as the bitfield's type (char, int, etc.). Bitfields don't span multiple container words, and container words don't overlap. On little-endian platforms, bitfields are packed LSB first, but on big-endian platforms they are packed MSB first within their container word. Alignment rules apply only to the container words.
If the instructions emitted and the instructions implemented both happen to match that, on every chip your code must run on, you got lucky.
If you want to produce same sequence of bytes regardless of underlying platform, then you have to do it by hand with uint8_t[] buffers and explicit shifts and masks. Casting pointer to struct to char* and writing it somewhere is inherently non-portable and this gas nothing to do with bitfields and nothing to do with things like __attributte__((packed)), although both of these things are useful when you want to do that and understand the (non-)portability implications.
Hopes, prayers, and a single version of a single compiler being involved.
A result of people avoiding declaring bit fields in serious use cases has been that compiler vendors didn't worry too much about bitfield codegen bugs.
Probably Gcc and Clang are OK on x86, by now. But that does not carry to, e.g., obscure microcontrollers. Heaven help you if your bit field members are supposed to correspond to hardware register sub-fields.