A Special Kind of Hell: intmax_t in C and C++
thephd.github.io
thephd.github.io
And on the other end of the spectrum, someone might well decide 16 bit is the intmax for an AVR8. Now your chrono code doesn't work at all.
intmax_t is as bad as int, for precisely the same reason. The developer has no idea what the capabilities of the type are. "Give me the biggest integer type you have" is not something any developer should say, ever, at least not in code that is intended to be remotely portable across time, space and architecture.
max(
static_cast<intmax_t>(
numeric_limits<T>::lowest()),
static_cast<intmax_t>(
numeric_limits<U>::lowest()))
numeric_limits<T>::min()
numeric_limits<T>::lowest()
I would put "char" in the running now that we (thankfully) live in an increasingly UTF-8 world, but at least it's more or less universally understood to mean an 8 bit entity (even though this is not strictly true). It certainly isn't "a character".
std::is_signed<decltype(1L * 2U)>()
?If sizeof(long) == sizeof(unsigned int), 1L * 2U results in an unsigned int, meaning the result is "false".
If sizeof(long) > sizeof(unsigned int), 1L * 2U result in a signed long, meaning the result is "true".
intmax_t exists probably because most integer types in C are not portable. You can't write C code that uses int64_t and expect it to compile, the type doesn't have to exist, neither do int32_t or int16_t. Say you have a generic int lowestBitSet(intmax_t) function and want to make it available, you don't care how large the input integer is as long as you do not truncate the largest type available to whoever ends up using it. Without intmax_t you end up with library developers having to either setting their own upper limit, limiting the portability of their library or checking for the availability of each type themselves.
intmax_t doesn't define an upper limit either, as TFA explains, because you don't know at runtime what the limit really is.
The idea for the library writer are that the bounds are large enough for whatever the user needs. User inputs a int64_t? Great the type intmax_t on that system is at least int64_t wide by definition. System doesn't have a int64_t ? Great the user can at best use a int32_t, which intmax_t would once again support.
> intmax_t doesn't define an upper limit either, as TFA explains
Yeah, it would almost be sad that C++ is getting bitten by Cs lack of name mangling. Almost because it is kinda pointless to define a maximum integer in a language where any programmer can add their own fixed sized integer types, so a generic library supporting arbitrary sized integers would have to take template parameters anyway.
And C99 requires long longs to be at least 64 bits wide.
Heck, I use 64-bit ints on 8051 cores that make an AVR look like something out of Star Trek.
I agree that intmax is silly, but I believe there are legitimate uses for variable-size types, like int_fast8_t and uint_fast16_t. These are their nominal size on x86 because the architecture has arithmetic operations for those sizes. But on POWER, they are both 32-bit types, because there are no 8-bit or 16-bit opcodes, and emulating overflow behavior with bitmasks incurs a performance hit.
The number of cases where it’s important to use different types on different architectures exists, but is very small.
That’s something that would be much easier to handle at the library-level for a finite set of architectures than at the language level for all archs.
I’d also like to see the elimination of printf specifiers. Another total disaster and source of soooooo many bugs. Especially with all of these variable types.
https://stackoverflow.com/questions/8500677/what-is-uint-fas...
But of course, ABI’s are important, since proprietary binary blobs are what is important.
They allow different compiler chains to interoperate, they're the basis of FFIs, there are so many situations where I have the source to both sides of some pipe-fitting operation and having a common ABI is what makes my job possible.
But making the standard library work with a changed ABI is the easy part. The hard part is every other library. Users expect to be able to compile libraries (both static and dynamic) and then keep using the binary artifacts even after upgrading their operating system. Admittedly, most libraries won’t use intmax_t in their public API, but if one does, changing the definition of intmax_t will cause the same types of ABI breaks as with the standard library. And unlike the standard library developers, those users probably won’t know how to fix them, nor appreciate being made to.
An int is 32 bits, a long is 64. Makes life much easier.
Just don’t do what Java does and make Integer a class and int a value type. Then you end up with references and the “integer cache” garbage. https://viveksb007.github.io/2020/06/integer-cache-in-java
The only thing that may need to deal with multiple supported ABIs simultaneously is the kernel. But maybe kernels are using more explicit types?
Personally, I don't like to write code with int/long/etc. I want to know how many bits are in the thing, because it usually doesn't make sense for the program to sometimes have one number of bits, and sometimes another (pointer/memory offsets excluded).
Right. Kernel ABIs use explicit-width types, and if an ABI change is needed, a new syscall is added. The old one continues to exist with the old ABI indefinitely.
When I ran into similar problems trying to distribute a binary for Linux users, I thought "Screw it, I'll just statically link everything." But the older statically linked C libraries would crash on newer kernels, and the newer static C libraries would refuse to run on older kernels. (Or do I have that backwards? Anyway, I ended up concluding that Linux doesn't really have a stable ABI.)
Now, if you think "operating system" == "kernel", then, yes this is a language problem. And yes, people who write in x86 assembly and directly invoke the Linux kernel don't have all of these worries. They have much more manageable backwards compatibility issues. But for most of us, "programming to Linux" means using the APIs defined in man pages.
Perhaps the actual "OS API" should be more completely divorced from C library issues. The kernel should define its own contract in some kind of IDL from which one can generate APIs for any language (including all of the variants of C that differ by implementation choices). Yes, it should map to C without any impedance mismatch, and the C folks can deal with how exactly it maps to different implementations of C, but other language runtimes shouldn't have to go through C and re-live all its problems.
Also given the target domain of C, the standard also must take into account possible free standing deployments, where an OS doesn't even come into play.
It's better to accept that C (and C++) are only portable in theory, in reality it's nearly impossible to write non-trivial code which compiles without warnings in a different compiler or runs out of the box on a different hardware architecture (still the language does a pretty good job at it everything considered). IMHO the standard committees should focus on the (handful) platforms that exist today, and not care too much about legacy or future platforms. The C standard shouldn't be some "pure ideal", just harmonize the differences between compilers and platforms, and only so far as it makes sense for real-world code.
Personally I'd rather see extensions from gcc and clang moved into the standard which are already battle proven instead of wasting time fixing 30 year old problems.
there's nothing that prevents you from using clang on all platforms, which will be exactly the same experience that if you used $LANG_WITHOUT_AN_ISO_STANDARD
void f(int n, ...)
{
float *t = malloc(n * sizeof*t);
// do stuff with t[i]
free (t);
}
void f(int n, ...)
{
float t[n];
// do stuff with t[i]
}
Notice that VLA are not more dangerous or risky than recursion. The exact same argument that you use against VLA can be used against recursion. Are you against recursive functions in C? Of course, you wouldn't traverse a long array using recursion, but if you are careful that the depth of recursion is logarithmic with the array size (e.g. as in binary search), then it is perfectly safe and reasonable to use recursion. The same thing with VLA. If you are sure that n is 2,3 or at most 4 there's no reason to avoid VLA due to stack scare, and they may make your code much simpler, clearer and better.
On the other hand, VLA are useful even if you do not use them to allocate memory. Pointers to vla can be used for many things, for example accessing a one-dimensional array (allocated by a single call to malloc) using two indices as if it was a 2d array.
Yes they are still there are optional, which basically means that other than clang and gcc, no other C compiler is going to bother supporting them anyway.
Google has sponsored a 2 year effort to remove all VLAs from Linux kernel, that is how much love they get from the security industry.
I understand that you have some sort of personal grudge against VLA, but there's no need to use misinformation to further your agenda. VLA are supported by all C compilers except Microsoft's. More precisely, they are supported by gcc, clang, icc, tcc and oracle developer studio (suncc), as well as several other experimental C compilers that seem to be written for fun as a learning experience.
I understand that it makes sense to avoid some language features on an OS kernel, like VLA or recursion. For example, it would make sense to avoid types of ambiguous size such as "int". This does not mean in any way that using "int" is wrong in all cases. Many algorithms (I think about mathematical and numerical codes) are very naturally expressed using these fancier language features.
You can make a very valid case for language simplification and against VLA and other "modern" features. But your argument is less powerful if you sound ignorant of the widespread support and uses of these features.
Also why Google's Kernel Self Preservation project for Linux has sponsored the 2 year effort that it took to clean up Linux from VLA usage.
You arguments for VLA adoption are meaningless when companies like Google and Microsoft are voting with their wallets against them, and WG14 has acknowledged their mistake.
Love and hate are emotional considerations not really worthy in this context. I would say that everybody that sees C as a valid replacement of Fortran loves and relies on VLAs and complex types. And there's probably much more people in the world interested in numeric codes than in "security" codes.
> that is why ISO acknowledged the mistake of placing them into C99 and removed them from the standard in C11.
This is most certainly not true. First, I'm almost sure that ISO never acknowledged any mistake with VLAs. Can you point me to where ISO acknowledged that purported mistake?
More importantly, your statement is false because ISO never removed VLA from the standard. That would be untoward and very contrary to their traditions. If they really wanted to remove them they would start by first deprecating them, and they are not deprecated. They are merely an optional feature of the language, just like the "main()" function is. The standard defines different kinds of conforming implementations, and some of them include VLA. Thus it is an official part of the language.
> WG14 has acknowledged their mistake
LOL. Funny that you mention that, since WG14 has added a whole new principle (the fifteenth) to their design document to acknowledge for VLA in future interfaces: http://www.open-std.org/jtc1/sc22/wg14/www/docs/n2086.htm
You should read the link contents before posting them here.
"In particular, the order of parameters in function declarations should be arranged such that the size of an array appears before the array. The purpose is to allow Variable-Length Array (VLA) notation to be used."
Notation for what?
"Application Programming Interfaces (APIs) should be self-documenting when possible."
And why is that?
"This not only makes the code's purpose clearer to human readers, but also makes static analysis easier. Any new APIs added to the Standard should take this into consideration. "
On top of that, if you bother to actually read the ongoing C2X mailings, there are no plans to bring VLAs back to life, besides adopting the syntax to describe function parameters.
Most devs that take security seriously actually take the effort to know the details of the devil that we have to deal with.
You are also spreading misinformation. VLAs being optional does not mean they have been removed from the standard as you keep claiming.
However they have backtracked on their position regarding supporting C on Windows, so latest MVSC does support C11 and C17.
sure, there's nothing wrong with self-inflicted pain. Rust shows that a compiler monoculture works much better in practice.
I thought the message was about everyone not your bubble.
It’s not that hard, unless you’re using -Wall or similar. And usually a platform that warns about something is probably giving decent advice anyways, regardless of whether the warning shows up elsewhere.
15 years and ditto. I've only used it and seen it used for printf with %j, because that sucks less than the C99 PRIxBLAH conversions.
> It's better to accept that C (and C++) are only portable in theory,
I don't agree with that at all, though.
Is this a practical concern? That seems like such a strong calling convention change that it wouldn't really bother me if you had to recompile all your software in this situation.
I mean, let's even imagine that there would be a way to make this change somewhat transparent and automated. You still end up with some types magically now taking 256 bits instead of (probably) 64bits before that. If you use this type a lot the performance implications are pretty huge, both in terms of actually doing maths with a 256 bit integer, but also for RAM and cache usage.
If you suddenly changed int to be 64 instead of 32bits you'd have a similar problem for instance. I don't think anybody would expect such a change not to require a full recompile. So what's the difference here?
I think you are spot on here. The OP's concern may be due to things like this:
> For example, let’s take a C Standard function that uses intmax_t, imaxabs
It looks like an unusual departure from typical libc functions like strtol(), strtoll(), etc. which have a primary type right in the name. To the untrained eye, intmax_t may look like some promise of portability, when it is actually no more portable than specifying int or long.
intmax_t is supposed to store any sized integer type. If you introduce a int128_t, it doesn't have any impact on 'int.' But it does mean that your intmax_t must be int128_t, and if it was 64-bit before, now your ABI changes (for intmax_t-using ABIs). Ordinarily introducing a new type or function wouldn't cause ABI incompatibility, but introducing a wider integer type that intmax_t cannot represent is not a valid choice.
I'd suggest considering ABIs with intmax_t in them unstable, and probably just avoiding use of intmax_t in ABIs entirely. Integer size-generic routines are best served with wrapper macros and use of the C11 _Generic() construction, or some of the typeof() magic used by, e.g., Linux.
Not that it should be much of a concern in practice, as far as I can tell intmax_t seems mostly about writing "generic" integer code without having to specialize for any int type, but that wouldn't make for a very clean or idiomatic public C API IMO. At least I don't think I've ever encountered it in the wild.
I think you're exactly right. That's what I was trying to suggest and address with: "Integer size-generic routines are best served with wrapper macros and use of the C11 _Generic() construction, or some of the typeof() magic used by, e.g., Linux." It predates the C11 _Generic() stuff, and the standards ignore the GCC extensions like typeof().
intmax_t val = imaxabs(val);
This is the first time `val` is defined in the block, so wouldn’t this raise a C compiler compilation error? If not, then I don’t understand the effect of this line.
intmax_t val = imaxabs(original);
for the snippet to make any sense.
int x = x;
Last time I looked at the speedup you got from loop optimization it at most a few percent for a very simple loop. For a loop that has some meat it'd be noise.
Wow, no. Most loop optimizations make loops significantly faster, even an order of magnitude is not uncommon. Write a few loops and compare the code generated by any compiler with O0 vs O3.
Lots of loop stuff also depends on non-overflowing signed integers. If integers can shrink on addition then you cannot safely do a lot of unrolling or fancier stuff based on affine loop transformations.
That is most definitely not true. All modern hardware is 2's complement arithmetic, and the corresponding signed integer arithmetic operation will silently overflow without complaint.
> Lots of loop stuff also depends on non-overflowing signed integers. If integers can shrink on addition then you cannot safely do a lot of unrolling or fancier stuff based on affine loop transformations.
Not really. Only certain particular patterns force weirdness on loops were overflow well-defined, and these aren't the most common pattern (a basic for (int i = 0; i < N; i++) loop doesn't need overflow to be undefined to have loop optimizations kick in). Even if you hit a case where it can be annoying (such as the poor man's multidimensional array access, because C lacks such niceties), you can multiversion the loop to fall back to an unoptimized loop in cases where overflow would have kicked in and otherwise use the optimized loop.
Where I've seen the most issues with signed versus unsigned is when LLVM decides to take a loop with a signed 32-bit index, promote the index variable to 64-bit (using zero-extension for all the related values), and then retain a signed 32-bit comparison for the loop bound. The unsigned version of the code had fewer problems. So much for signed overflow being good for optimizations.
But not the same width. Does it overflow at 32bits or 64? If you define it as one or the other, that requires a runtime check.
I do not know who was against man pages, but God, man pages are a God-send. It must stay. I never liked info though.
---
In any case, I write in Forth, Ada, Go, and JavaScript (Vue.js). Not much C these days. If someone asks me a simple program to write to extract data from a format specified by him, C is what I would choose; mainly: structs. I could do it in Forth, but easier in C, he has GCC, I can create a binary easily, etc.
Second, in my personal opinion, it's really much safer to use C89/C90, unless you have very good reasons to use the extensions present in the newer versions of the standard. Even large projects like SQLite appreciate this approach. And if you are developing for an environment supporting C99 and newer standards, you could as well use C++ (or a newer language).