A guide to better embedded C++
mklimenko.github.io
mklimenko.github.io
1: Don't use bitfields when interacting with registers. The language does not guarantee the behavior you want. Or even the behaviour you think you want.
2: 'volatile' is the contract that we (language users) have hammered out with the compiler industry over the last 20 years; it means "don't get smart with this memory location". Use it.
3: For any SoC of more than trivial complexity, the vendor will supply headers that describe the hardware (register names and addresses, field names, sizes, offsets, enumerations, etc.). You will hate their naming convention. You will hate the way they encode field sizes, offsets, masks, etc. Deal. Because the alternative is you, or your intern, making dozens of mistakes attempting to transcribe from the documentation. Here's a hint - the better vendors auto-generate these headers from the VHDL. They are often right even when the documentation isn't.
4: It's trivial to template most interactions with registers. See, for example: https://github.com/steffanw/laks
5: A little investment in your abstractions, even if the lower levels look a bit more complex or verbose, helps make your application logic easier to follow.
The discussion here about re-ordering is largely moot; if you're having issues with the core re-ordering your transactions, either your core is børked or you haven't mapped your peripheral space correctly. The former is pleasantly rare, the latter common enough that your compiler vendor is going to ask you about it before they give you a bug number.
Here was a great summary I found from a comment explaining the dangers of bitfields.
Seeing it spelled out like this made it clear which to choose, lest you're a fan of nasal demons :).
"The royal mistake is to use bit fields in the first place. The following is not specified by the standard:
– whether bit field int is treated as signed or unsigned int
– the bit order (lsb is where?)
– whether the bit field can reach past the memory alignment of the CPU or not
– the alignment of non-bit field members of the struct
– the memory alignment of bit fields (to the left or to the right?)
– the endianess of bit fields larger than one byte
– whether plain int values assigned to them are interpreted as signed or unsigned
– how bit fields are promoted implicitly by the integer promotions
– whether signed bit fields are one’s compliment or two’s compliment
– padding bytes
– padding bits
– values of padding bits.
– and so on."
Source: https://embeddedgurus.com/stack-overflow/2009/10/effective-c...
There's a bit of nuance here though. The function that the author chose to reimplement provides the platform-specific implementation of a function that's actually specified as part of CMSIS (https://developer.arm.com/embedded/cmsis). Its portability across cores and MCUs is not much of an issue, and you're usually tied to the compilers in this sort of projects, too. So it's not necessarily a bad idea from a purely technical standpoint.
That being said:
- No one who writes embedded software for a living expects to see bitfields in their source tree. In fact, there are standards that this code would break (e.g. in CERT C: https://wiki.sei.cmu.edu/confluence/display/c/EXP11-C.+Do+no... ), so it wouldn't even get well through code review. It's a bad idea to surprise your coworkers :-).
- Implementation-defined behaviour can sometimes be surprising and non-uniform. More often than not, the distinction between "bug" and "implementation-specific" is very blurry, and the corner cases are very hard to catch. You may think you're OK because your code doesn't need to be portable and it works fine on your device, but then it turns out the memory alignment of bit fields varies with field size because of course it does.
- Precisely because of their infrequent use, the implementation of bit fields is one of the things that vendors often get wrong in early versions of a toolchain. At best, it's just a matter of inefficient access. At worst, it's bugs. And if you think that can't possibly go wrong, let me tell you about that time when an early version of msp430-gcc translated access to volatile variables into a NOP.
(Disclaimer: that was a long time ago, on an early port. The MSP430 port of GCC is actually very solid.)
So basically, just don't. Don't. There are well-known idioms for bit access. They are not readable as in "someone who knows English can read those programs just fine" but it takes like two minutes to understand those idioms. They are very readable, as in "anyone who's ever made a LED blink understands them". Yes, it's domain-specific knowledge, but you're already twiddling bits on tiny machines in a language that's not English. You're already full of domain-specific knowledge.
Edit: ah -- just to make sure this doesn't get interpreted the wrong way. IMHO, C++ in embedded systems is not a bad idea. I don't use it because I've never worked on a project where I needed it, and therefore I never got the chance to be proficient with it. But I've seen solid, reliable devices running (what, to my untrained eye, seemed like) very clean, debugable C++ code.
Also, "embedded" refers primarily to purpose, not platform capabilities. There are plenty of embedded platforms running on eight-core Xeons and state-of-the-art DSPs and whatnot. It's not all tiny MCUs with 4K of RAM and no MMU.
Probably will work with a specific compiler. If it fails it'll likely fall on it's face fail. Which with embedded is the best type of failure.
Also you probably can force the behavior you need with pragma's (not portable)
Portability is usually completely unimportant in embedded.
C89 just don't.
C++ in embedded. Lord god no, if you want to do that use rust instead. Or embedded python or lua, anything but C++.
Note so sure, maybe depends on what 'embedded' exactly means to you. I mean, years ago I was working on some TI C6x series DSPs and needed a circular buffer. Hadn't one in C, did have a templated one in C++. Didn't think twice: if the compiler would swallow it, and it did, why would I go through the hassle of doing the work again in C possibly beeding different implementations for different type or a bunch of rather ugly void* stuff? Everything worked out. So for that particual case, C++ in embedded, god yes, as it meant: less development time, reuse existing implementation both for float and int types, etc. And that's +10 years ago so I'd imagine there's now more compilers which would do it.
First Rust needs to reach C++'s tooling maturity on embedded space.
When there is High Integrity Rust certification, compiler backends for all major CPUs, SDK support on OSes like INTEGRITY or Mbed, UI embedded tooling for IoT displays, maybe we can start seeing production code there.
Yeah and that's soon. Anyone starting an embedded project in C++ is making a grave mistake.
> Mbed
Those guys make a bad mistake and will pay for it.
It took C++ about 20 years to reach maturity in this space, with help of several OS and compiler vendors.
You might be willing to bet the house on Rust, most business aren't, not in the current state of its toolchain.
In 10 years from now, yeah I can clearly see it.
Java is a partial replacement for C++ that isn't quite good enough for desk top applications. Hence C++ is still alive in that space.
The JRE was a anvil that language grabbed a hold of, which set them back about 10 years. because every language that depends on the JRE also suffers it's limitations.
And the web diverted resources away from native programming for 20 years.
However in the last 5-10 years we've seen a couple of well designed native compiled languages being developed. And those are going to replace C++.
And LLVM project and work done on KIT compilers provides a strong alternative to the JRE for language development.
Seriously, go is better than C++ now. And Rust is very rapidly getting there.
However, C++ was going strong in all relevant desktop environments, OS/2, Windows and Mac OS. Even on commercial UNIX systems everyone was jumping into CORBA as the next big thing, and I surely wouldn't ever bothered to touch CORBA with plain C stubs.
Java had a lost opportunity with Sun being religiously against AOT compilation, an option only available via commercial JDKs, mostly for embedded deployment.
Graal and SubstrateVM are changing that, but it will still take a couple of years to actually change everything. Project Metropolis calls the Java on Java plan, the next 20 years.
Not only are those web browsers written in C++, throughout those 20 years C++ has been present on web pages via plugins and now WebAssembly.
We cannot have LLVM and kill C++ as well, unless you are planning to rewrite into something else.
Go is a better C, but definitely not something worthwhile against C++.
Using Git urls as package names and writing generic code as if using Borland C++ 2.0 for MS-DOS with BIDS pre-processor macros is definitely not something enjoyable.
Rust is improving at a good pace, but until it cuts its dependency on LLVM it will always depend on C++ tooling.
Love the language, but suggesting that it is at the same tooling level as C++ to business using it for the last 20 years is just irresponsible.
There are lots of domains, even on embedded where Rust eco-system just isn't there nor does it seamlessly integrate with existing workflows.
Using a programming language is much more than just installing a command line compiler and hacking away on a text editor.
It took C++ almost 20 year to replace C in many high performance domains, and for better or worse, it is mostly copy-paste compatible with C89.
Any language that wants its place needs to account for similar uphill battle in all domains.
- LLVM backend is limited regarding the supported CPU architectures;
- Rust compiler is not certified for high integrity domains, where human lives might be endangered;
- Features like SIMD are still nightly
- Lack of integration with closed source toolchains from OEMs for mixed debugging and other development workflows, e.g. Keil, Microchip
These issues might be complete irrelevant, or showstoppers, depending on the company.
Next step is https://doc.rust-lang.org/beta/std/simd/index.html but you can use https://crates.io/crates/stdsimd in the meantime to try it out, as they're the same.
Stuff like https://crates.io/crates/faster is already built on top of this and is just waiting for the initial intrinsic stabilization, too.
Yes! And the hateful naming might come straight from the RTL where digital designers have their own notions about what is a pretty name ;-)
> They are often right even when the documentation isn't.
More so -- if you decide for changing these names, one day debugging a complex issue will require talking to RTL people and everybody will be instantly lost -- your names won't be meaning much to them.
To back this up: it's important to remember that reads and writes to memory-mapped registers have meaning independent from simple memory access.
I once was writing an interrupt handler for an MCU. There was a bitfield register that indicated which interrupts had fired on read, and acknowledged/cleared an interrupt on write (IIRC). I was reading the register, clearing the interrupt I'd just handled, then writing it again. So I might read 0x84, mask out 0x80, then write 0x04.
Or so I thought. Turns out what the processor actually wanted was a direct write. You'd just write 0x80 to acknowledge that interrupt, not read/mask/write. And reading had a different effect on the MCU too, as I recall. So I was only getting interrupts sporadically and my code didn't work.
Now add compiler bitfields. You have to be 100% certain that the implementation behaves the way the CPU wants (no extra reads/writes, expected bits set/cleared), and that it won't change from version to version.
Instead it's common to use macros to deal with registers. Not quite as pretty as bitfields, but much more predictable.
Instead, we just checked return values of everything.
However, this might change in C++23. Herb Sutter just proposed C++ switch to deterministic exceptions. The paper is quite good: http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2018/p070...
It looks popular, so there is a high chance it will become a thing allowing stl to work in the embedded world.
There's still the downside of course - looking at the code to 'prove' it is troublesome. And often the code generated for bitfields is ugly. You can likely generate better code by hand.
So the single remaining advantage is, the C looks pretty clear. There's the register; there are its bits by name; assuming that's all been checked to work right, the coding (and code reading) can proceed without any more concern over bits.
ROFL!
I've been working in embedded for over a decade, and it's always quagmire of buggy hardware patched over with C code, half of whose programmers have actually transitioned over from hardware design and don't know what they're doing.
With that in mind, I'm going to assume the author was sarcastic.
I love the field, for getting to work deep inside and very early on bleeding-edge hardware, but by god most days feel like being in the coal mine of computer systems.
Rust is promising, but will never be a good option on low level devices.
I have seen some nice stuff done with Forth
Why not?
The system before that was largely just a USB driver, an event loop and some interrupt handlers, supporting a large amount of ported-over and mostly already existing stuff in C that did the interesting and product-defining algorithmic heavy lifting. I guess that we could have written the infrastructure in Rust, but it's hard to see the win in using two languages (the algorithmic stuff was not going to get a rewrite).
Rust isn't great for other reasons, too. Finding good embedded systems programmers is hard enough; require something bleeding edge like Rust and the population of candidates plummets to "well, we might be able to find and hire one person this year" territory.
The CPU architecture is more likely a problem. And, as you say, the stuff that’s not purely a technical requirement. We’ll get there!
http://ww1.microchip.com/downloads/en/DeviceDoc/41236D.pdf
PIC12F508.
It did a number of things. It had logic for controlling power on a low power device, coordinating between a number of other micros. It also acted as a last ditch watchdog system that triggered a recovery of the device
*(volatile std::uint32_t*)reg_name = val;
Some proper C code will define the 32-bit register like this (usually in a header provided by the hardware vendor, generated from the HDL):
typedef union some_reg {
uint32_t raw;
struct {
uint32_t bits1:4;
uint32_t bits2:3;
uint32_t rsvd:5;
uint32_t bits2:20;
}
} some_reg_t;
some_reg_t reg;
reg->raw = READ_REG(reg_addr);
reg->bits1 = 4;
WRITE_REG(reg_addr, reg->raw);
volatile some_reg_t *reg = (some_reg_t*)reg_addr;
reg->bits1 = 4;
Believe or not, a majority number of ARM MCU vendors does that with CMSIS headers in my experience (ST, TI, NXP, Nordic, EFM32...). To be more precise, the registers are accessed using mask and shift macros. I've work on many ARMs too but only Atmel's header files use struct bitfield you mentioned.
But yeah, I've also seen that most of non-ARM vendors use struct bitfield.
That, and because the C standard leaves it implementation-defined.
*(volatile std::uint32_t*)reg_name = val;
struct DeviceSetup {
enum class TableType : std::uint32_t {
inphase = 0,
quadrature,
table
};
std::uint32_t input_source : 8;
TableType table_type : 4;
std::uint32_t reserved : 20;
};
volatile auto device_registers_ptr = reinterpret_cast<DeviceSetup*>(DeviceControlAddress);
For the second case, most of it goes away. You still need to know exactly how the value you set is interpreted, but you get a readable description of each part and you don't have to think of bit shifting every time they're set.
The declaration is longer of course. But what would you prefer to see at usage time:
MREG42 = MREG42 & table_type_mask | inphase_val
device_setup_registers->table_type = TableType.inphaseWhich does not do what the author wants it to do.
volatile auto device_registers_ptr = reinterpret_cast<DeviceSetup*>(DeviceControlAddress);
The first possible bug is aliasing. The type of reg_name and DeviceControlAddress are not given, so it is possible there isn't a bug.
The second bug relates to memory ordering. Adding "volatile" will tell the compiler to do things in order, but the compiler will not pass that requirement on to the CPU. The CPU itself may reorder things. I saw this affect an embedded system back in 1998, and the problem has only become more common in the 2 decades since. Generally you will need assembly code to avoid the bug.
int64_t val = (*reg0 << 32) | *reg1;
int64_t val = reg0 << 32;
val |= reg1;
You can still hit the problem. In my case, the code ran fine until we got an upgraded CPU. (from MPC6xx series to MPC74xx series)
Suppose you store to registers at 0xf0000ffc, then 0xf0000104, then 0xf0000ff8. You need the stores to happen in that order. The instructions execute in that order, creating 32-bit chunks of data headed out toward the memory bus. There are multiple write buffers however, so they can go in parallel, each getting a distinct write buffer. They then head out onto the memory bus in some randomish order determined by timing issues internal to the CPU.
In my case, I had to add "eieio" instructions between each pair of accesses for which ordering mattered. FYI, that is a real instruction, supposedly meaning "Enforce In-Order Execution of I/O".
The ARM architecture does this more sanely. Page table entries have a flag that lets you choose between regular "Device" memory or "Strongly-ordered" memory; the latter performs all memory accesses in order without needing any synchronization instructions, and is more convenient in simple situations.
[1] https://www.nxp.com/docs/en/reference-manual/MPC7410UM.pdf - Table 3-8
The proposed alternative of using a struct appears more confusing at first, but it's really helpful when referring back to the register layouts. TI makes pretty good usage of the method for the co-processors in the BeagleBones [1]. Generally I think it's nicer to have a `settings->control_bit = 1` than `*(base_prt + control_bit_offset) = 1`. Though the whole `reinterpret_cast` and volatile stuff in C++ confuses me everytime. C is _much_ easier but less type safe when dealing with volatile.
1: http://processors.wiki.ti.com/index.php/PRU-ICSS_Header_File...
> Generally you will need assembly code to avoid the bug.
What do you mean by this? Compiler outputs binary code the same way assembler does. If the CPU reorders your instructions, it will happen in both cases.
If there are some specific instructions which create memory/reordering barriers, they can be expressed in higher level languages as well.
__asm__ __volatile__("eieio":::"memory");
Normally the CPU will freely reorder loads and stores that go to different addresses. It is not necessarily reordering instructions; the reordering happens on the way to the memory bus.
That special instruction helps. It will ensure that previous accesses to IO memory will be done before any that follow. It also has a similar effect in RAM, but sadly not between RAM and IO memory. For that, which might be needed for a DMA engine, you'll also need the "sync" instruction.
As of 2011, you can. In C++ it’s with std::atomic. http://en.cppreference.com/w/cpp/atomic/atomic C11 offers comparable tools, I believe.
For example, consider two values written to the same location. The compiler is free to optimize out the first one because no valid threaded program can depend upon seeing that value briefly appear.
I'm not 100% sure what you mean by your second paragraph. I think you are saying that repeated writes to a `std::atomic` can be optimized out, which sounds like it is true since `std::atomic` is not `volatile`. https://stackoverflow.com/questions/45960387/why-dont-compil...
PowerPC does treat it like a memory barrier instruction (eieio - Enforce Inorder Execution of I/O), but that's a stronger guarantee than you'd want for all volatile accesses.
Likewise, you don't need to keep memory free for other applications to use, so there's a strong tendency to just allocate everything statically. It's simpler and more reliable.
Grabbing realtime priority or disabling interrupts with a lock_guard so you can't forget to release is a god send.
The compile-time features of C++ are a great reason to switch over though, and for devices with greater resources, C++ becomes an even better fit.
.... there are 0 penalties associated with RAII.
struct my_interrupt {
my_interrupt() { vendor_specific_horrible_stuff(0x237ABC); }
~my_interrupt() { if(whatever) vendor_specific_horrible_stuff(0x237BAC); }
};
void f() {
my_interrupt is_enabled;
// do stuff
}
void f() {
vendor_specific_horrible_stuff(0x237ABC);
// do stuff
if(whatever) vendor_specific_horrible_stuff(0x237BAC);
}We do disable RTTI and exceptions though.
The problem with C++ is it has a few language features that are not deterministic making standard C++ not ideal for real time code. Most of the stl uses these features, so most vanilla C++ is disabled for many embedded environments.
Also, there is a push for C++23 to have deterministic exception handling. If this passes (and it likely will) most of the stl will become available for embedded, and when that time comes C++ is going to become far more of a viable option than it currently is.
If you're curious you can read about it here: http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2018/p070...
The selling point is features like RAII and templates, which can greatly reduce duplication and improve readability. Sure, you technically can't use the STL for whatever reasons, but practically it doesn't actually matter at all, and its absence comes with the territory for everyone involved.
Being able to use more of the STL in contexts like this would be nice for future projects, perhaps.
Check out MISRA C, and the JSF C++ Standard for examples of embedded code standards.
It doesn't take having much more resources before switching away from C++ is a vastly better option though.
What if it's bits 13 through 18? An unaligned 16-bit access? Two 8-bit? A 32-bit? Where is this specified?
Depending how your peripheral is implemented, this absolutely can matter. Nothing stops you from building devices where the low byte of
*((uint32_t *)x)
*((uint8_t *)x)
This is even before considering stuff like a GPIO output implemented with two write-only registers, where 0 leaves the bit unchanged and 1 sets/clears it. The result of layering a bitfield on that does not seem intuitive to me.
The article also says that you should build abstraction layers for your peripherals, instead of making all your code write to registers everywhere. That seems (a) pretty well-settled as true, and (b) entirely independent of whether you do the register accesses with bitfields or explicit shifts and masks.
Admittedly, a compiler is free to implement a dereference of uint32_t by loading four separate bytes, too, hence the imprecise "very undefined" in the first line...
I see the makings of a fun trick to play on a colleague. I don't see anything I'd be too inclined to use in production code.
I was worried people would get too fancy with C++. Probably it would have been fine if we would have taken away most of the language.
It turns out the problem with C in that setting was that good C code seems to require a lot of ceremony and conventions for things like error handling and modularity.
I hope that Zig (https://ziglang.org) will become a viable alternative in that space. It's quite a small language, more comparable to C than to C++ or Rust. It's meant to result in similar LLVM bitcode as well-written C, while being highly compatible with C and and giving you some extra safety guarantess over C as well.
I am not affiliated with the project, only supporing it on Patreon, so I cannot speak about the designer's intentions with Zig specifically.
Generally in language design, if something works the same way as in other languages, that's great. You can re-use the keywords/syntax and build on peoples prior knowledge. However if you have some feature that actually works quite a bit differently from other languages the syntax should probably be different. Otherwise people might feel at home right away on a superficial level, but actually get bitten by the surprising ways in which the new thing works differently to the one they are used to later.
And quite frankly the abstractions given in the article are quite trivial. Any decent programmer would create similar abstractions as a part of normal course of work because decent programmers know they are forgetful and they want to spend their modest number of brain cycles on the higher levels of thinking instead.
This is incorrect. Compilers are allowed to reorder and combine atomics as long as they follow the rules. See: https://github.com/jfbastien/no-sane-compiler
SERIAL_TX_DATA = 0x01;
SERIAL_TX_DATA = 0x02;
#define SERIAL_TX_DATA (*((volatile uint32_t *)<addr>))
I think on an ARM processor if it's normal memory you can't guarantee ordering. But ARM supports a device memory type which disables caching and reordering memory accesses by the hardware.
http://infocenter.arm.com/help/topic/com.arm.doc.dai0321a/DA...
Observation: CS appears to teach students that code with side effects is evil. AKA hide you side effects behind os calls. With embedded code side effects are important.
I much prefer having some kind of READ_REG/WRITE_REG inline function (or macro at a pinch). Sooner or later you're going to find yourself wanting to log all register accesses, or run the firmware against a model of the hardware, or something, and when you do it's a great help to have to handle it in only one place.