When ‘int’ is the new ‘short’
googleprojectzero.blogspot.com
googleprojectzero.blogspot.com
For as long as I can remember I've just used size_t, uintptr_t, uint32_t/int32_t (or any of the 16/32/64 variants), exactly because I want to be explicit about the machine word sizes I'll be dealing with. Before that, I always used similar (u32/i32, LONG/ULONG, ...) platform-specific typedefs on proprietary systems too.
For all practical purposes, int/unsigned int has been at least 32 bits since the early 90's (well, on modern platforms) but why use those if you can explicitly declare how many bits you actually need.
(I've bumped into a few archaic platforms where stdint headers weren't present but it's easy to just add a few build-specific typedefs somewhere in that case.)
One case changing an index in a for loop to an int, code went from 34024 bytes to 34018 (saved four bytes). But changing uint8_t i, j, k; to uint16_t i, j, k; code compiled to 34068 bytes, gain of 44 bytes.
"Num: %" PRIuFAST32 " found\n"
"Num: %d found\n"So, what if Clang implements it, but not GCC? Or what if Clang and GCC implement it, but not the Sun or Intel compilers? Or what about all the GCC copies every board maker forks when they create something custom?
It's tricky making in-compiler behavior non-standard (though, I guess that's what compiler flags are for).
(Type-safe at runtime, that is, via automatically passing a type info argument along with each format argument; C++ is braindead enough that doing the check at compile time is only somewhat possible with C++14 - as far as I can tell, it only works if you define the format outside of a function - and probably fairly slow, since that involves a template instantiation per character in the format. Oh well.)
I'd like to see that.
I can see some pretty straightforward ways to do it with C11 and _Generic, but I don't see how C99 helps.
Basically, if you are still typing "unsigned long long" or even "long" in 2015 in a modern environment, please stop. But—you may say—we want long to be 32 bit on 32 bits and 64 bit on 64 bits! No, you don't. That makes your system difficult to reason about. Plus, you'll probably start casting your inputs to printf() instead of using proper printf type macros, which breaks things even further. Adopt proper types, use proper types, stop programming like it's 1975. Good luck.
Rust allows silent truncation of values in numeric conversions and considers it "safe", because other features for memory safety will catch buffer overflows — but it doesn't care about cases where the program will do a memory-safe logically-invalid thing (e.g. write to a wrong location within a buffer).
That's because Rust has no integer size promotion at all, which means `len as usize` and `len as c_int` are required all over the place when interfacing with C (and the `as` operator has no overflow checking by design).
I do agree that "as" should have range checking. Some of it can be done statically (e.g. converting to a strictly larger type, and when the range of possible values are known) and the rest can be done with minimal overhead if they emit the code right.
This may lead to some unnecessarily verbose code in some places but it leaves no surprises. It also gets rid of the need to do dirty tricks like union casts or pointer tricks when you want a bitcast from float to int or vice versa which is common when you write SIMD code, for example, using bitwise ops to extract the sign bit of a float or something like that.
The problem is that `x as usize` will compile without warnings to something I didn't intend and causes bugs if usize is smaller than the type of `x`.
AFAIK I can't avoid casts to `usize` any other way than using `usize` for almost every integer type in Rust. To make things worse my main use case for Rust is interfacing with C code, which means I have to deal with other types as often as `usize` and end up with these risky casts in almost every expression! It's awful.
What I'd prefer is something (it could be another operator, but I'd prefer promotion to keep syntactical noise low) that would either compile if typeof(x) <= usize, or would not compile at all (i.e. if I write code that is accidentally 64-bit only, I want it to fail to compile on 32-bit machine, instead of merrily compiling to something that is buggy and will corrupt the data or even be exploitable through FFI which requires these casts that become unsafe).
#[cfg(target_word_size="32")]
struct ThisCastIsUnsafeFixIt;
#[cfg(target_word_size="32")]
const u64_to_usize: ThisCastIsUnsafeFixIt = ThisCastIsUnsafeFixIt;
#[cfg(target_word_size="64")]
fn u64_to_usize(x: u64) -> usize { x }
...
u64_to_usize(123) // won't compile on 32-bit
I guess I could (thanks for the tip - I didn't know about this cfg(), I've been trying with traits and sizeof::<>).
But I'd prefer it to be in the language:
• lossless integer type conversion seems like a very basic problem to me, that shouldn't need programmers to fix it themselves in a custom way in every crate.
• even if I fix it in my code, I'm still worried about other people's code, because I assume that they also develop on x64 and unintentionally write casts that are subtly broken on smaller architectures.
Also, size_t is a frustrating beast. It's meaning is dependent on the platform. The Single Unix spec only calls for size_t to be an unsigned integer type. Now imagine you're writing code to compile over multiple mobile platforms as well as on x86_64 on the server side. Can you tell me what is the largest number you can address with that type -- without getting into a long google/stackoverflow session or hitting the compiler manuals for each of those platforms? If you absolutely want to make sure that your type can handle the values you expect it to handle, better give it well defined types provided by stdint.h (uint32_t is sooo much better than just int or unsigned int or even size_t for this purpose).
Now granted, you'd need to interact with external libraries (including libc/libc++) that'll want to use size_t etc. Not much you can do here but be very careful when passing data back and forth between your code and the library code. But that's been the lot of C coders since time began.
All you need to care about for cases like these, when you're talking about the size of something, is that both malloc() and new[] handle allocation size using size_t.
That, to me, says pretty clearly that "the proper type to express the size, in bytes, of something you're going to store in memory is size_t".
It can't be too small, since that would break the core allocation interfaces which really doesn't seem likely.
You don't need to know how many bits are in size_t all that often, and certainly not for the quoted code.
I don't know why you are downvoted, but this is very important.
Never send anything "on the wire" (or to a file) unless you know its exact size and endianness.
However for cross platform support using size_t in an API (as in what is exposed via .dll or .so) is a must. It's exactly the correct way to write cross platform code.
If you have no requirement that says otherwise, you should have an explicit marshalling and demarshalling steps that transform your live data objects into opaque BLObs. It would be highly desirable if your BLObs have some header that contains metadata to be used exclusively for marshalling purposes, at the very least size of the payload, object type id and format version id will save you lots of trouble.
Now what happens if you need high performance and are willing to trade of code complexity for faster execution. You can just copy your native object's bytes into the BLOB payload, just as long as you can correctly identify the source platform's relevant characteristics in the header. Then when the target host does the demarshalling step, it can decide if the native format is compatible with it's own platform and just copy the payload into a zeroed buffer of the correct size. If that its not the case, it will have to perform and extra deferred marshalling step to put the payload in "canonical" format prior to demarshalling proper.
You can even make the behavior configurable, so that customers running an heterogeneous environment do not suffer a performance hit for the sake of the customers in homogeneous environments.
The point is that a lot of the times, the two machines on either end of the wire need to agree on sizes of various fields you're sending (say in protocol headers). And then you want to work with that data internally in the code on either side. You better be absolutely sure how many bits you have in each type that you're allocating for these purposes.
And going even beyond that, very common, use case -- a lot of code reads cleaner and lends itself to debuggability when you know the exact sizes of the types you're using. It's not something reserved for just network programming.
In your third paragraph, i agree on the debuggability front (if you are actually reading memory dumps, otherwise, why should it matter). About the code reading clearer, I guess this is more a matter of taste.
Why does this matter? size_t is intended to be used as an index into a dense array, i.e. for every Index i you may want to store in a size_t, you also store i elements X of some data type. Since that number is limited both by the software architecture and the hardware available at runtime, why would you want to know exactly how many X you can store?
Because you want to communicate with other systems. Not just the system your code is running on at that moment.
You can never ever have a buffer bigger than what size_t allows. If you do then you're no longer talking about the C programming language as it explicitly breaks the C specification.
> Now, the more astute reader will point out that I just sent over 4 gigabytes of data over the internet; and that this can’t really be all that interesting - but that argument is readily countered with gzip encoding, reducing the required data to a 4 megabyte payload.
This was pretty much my first thought on seeing the IOBuffer signature - "That exploit payload is going to be huge". But things are not always as they seem and using gzip to generate a large string on the client is something I had not previously considered.
Good spot though; I kind of doubt that this was a conscience design decision and probably just a slip up.
And agreed, good catch.
And here we have a prime example of Chrome, a major cross-platform project with high visibility that is not using these types nor didn't define their own in this case. These types were intended to help reduce mistakes. But compiler fragmentation basically resulted in organizations avoiding it, leading to these types of mistakes that could have been avoided in the first place.
Maybe I'm reading this wrong, but to me it seems like this is saying go ahead and use the fixed size variants whenever, but it is still OK to use int when you need <=32 bits.
"<stdint.h> defines types like int16_t, uint32_t, int64_t, etc. You should always use those in preference to short, unsigned long long and the like, when you need a guarantee on the size of an integer. Of the C integer types, only int should be used."
Still, at least nowadays there is no excuse -- everything from TI to VC++ supports stdint variations.
In any case, it isn't as if the language doesn't allow for type alias.
EDIT: looking at the documentation, it appears that __int8 is supposed to always be an alias for `char`, even as far back as 2003: https://msdn.microsoft.com/en-us/library/29dh1w7z(v=vs.71).a.... However, the workaround found in msinttypes suggests that Visual Studio 6 does have this problem. I weep for those still using it.
Now; on x86_64, with gcc and clang an int is still a 32-bit integer type;
The bug example there, sure, needs a signed type, so you can't blame the type for its wrong usage
I've had more bugs coming from using signed types that I'll not be bothered by writing 'unsigned' ever again
std::size_t i = foo.size();
while (i != 0) {
--i;
...
}It's also worth pointing out the style of loop they give can't be used at all if you are counting down iterators or pointers instead of numbers.
(You can't thus check for int overflow by checking for wraparound, see http://c-faq.com/misc/intovf.html)
Of course, but as you say it yourself: if. It seems a bit too general to abandon unsigned completely because there are cases where it is not appropriate. By that logic there wouldn't be much types one can use at all.
That's useful if your language lacks the ability to return multiple values from a function. That typically was (probably still is on quite a few architectures) the case for languages designed for speed.
Also, in C, int was implicit (see https://github.com/mortdeus/legacy-cc for example source code). So, using int made your programs shorter. That's important if your multi-user system doesn't have much memory (the first PDP-11 that ran Unix had 24 kilobytes of memory), and if you like concise porgrams, as Ritchie apparently did.
Chars: I hate it that they are signed because I like the convention of promoting them to int, and then using -1 as an error. It's easy to forget to convert to unsigned first, and the compiler will not complain. In the past I've used 'unsigned char' everywhere, but it's a mess because strings are chars and all library functions expect chars. My new strategy is to use 256 as the error code instead of -1. The only problem is that getchar() uses -1, so it's weird. IMHO, it's a C-standard mistake that char is signed.
I used to use int for indexes and long for file offsets. But these days, int is too short on 64-bit systems and long is not large enough on 32-bit systems.
ptrdiff_t is the new int. I've switched to ptrdiff_t in place of int and off_t in place of long. Ptrdiff_t is correct on every system except maybe 16-bit MS-DOS (where it's 32-bits, but I think it should be 16-bits). Off_t is a long long if you have '#define _FILE_OFFSET_BITS 64'. Ptrdiff_t is ugly and is defined in an odd include file: stddef.h. It's not used much by the C library.
The C library likes to use size_t and ssize_t. The definition of ssize_t is just crazy (it should just be the signed version of size_t, but it isn't).
I understand why size_t is unsigned, but I kind of wish it was signed. It's rare that you have items larger than 2^(word size - 1), so signed is OK. You are guaranteed to have -Wconversion warnings if you use size_t, because ptrdiff_t is signed (even if you don't use ptrdiff_t, you still get a signed result to pointer differences so you will have warnings). Anyway, to limit the damage I make versions of malloc, strlen and sizeof which return or take ptrdiff_t. They complain if the result is ever negative. Yes this is weird, but I think it's better than having many explicit casts to fix warnings. Casts are always dangerous.
One knockon effect of this is that strcmp() will return different values on the two different platforms for UTF-8 strings (because 0xff > 32, but -1 < 32)...
Incidentally, I don't know if you know about intptr_t; it's an int large enough to put a pointer in losslessly. It's dead handy. (My current project involves a system with 16-bit ints, 32-bit long longs, and 20-bit pointers...)
I try to be conservative with the definitions I use, so I'm worried that intptr_t might be too new.
The standard doesn't specify whether char is signed or unsigned, it's left to the implementation.
I've noted some compile time and run time checking options at:
http://www.pixelbeat.org/programming/gcc/integer_overflow.ht...
It's not so bad. It means you can be really consistent about returning error codes from every function in the exact same way. This is one of the very few things I actually like about the Win32 API. If only they used the same type of error codes in every section of the API.
Also note that C only half-heartedly supports objects larger than SIZE_MAX/2: relevant quote from http://en.cppreference.com/w/cpp/types/ptrdiff_t
"If an array is so large (greater than PTRDIFF_MAX elements, but less than SIZE_MAX bytes), that the difference between two pointers may not be representable as std::ptrdiff_t, the result of subtracting two such pointers is undefined. "