Should I Use Signed or Unsigned Ints?
blog.robertelder.org
blog.robertelder.org
However, since apparently we've collectively decided that we're always operating in ring 256/65,536/etc. instead of the real world where we only operate there in rare exceptional circumstances [1], instead of an exception being generated when we under or overflow, which is almost always what we actually want to have happen, the numbers just happily march along, underflowing or overflowing their way to complete gibberish with nary a care in the world.
Consequently the answer is "signed unless you have a good reason to need unsigned", because at least then you are more likely to be able to detect an error condition. I'd like to be able to say "always" detect an error condition, but, alas, we threw that away decades ago. "Hooray" for efficiency over correctness!
[1]: Since this seems to come up whenever I fail to mention this, bear in mind that being able to name the exceptional circumstances does not make those circumstances any less exceptional. Grep for all arithmetic in your choice of program and the vast, vast majority are not deliberately using overflow behavior for some effect. Even in those programs that do use it for hashing or encryption or something you'll find the vast, vast bulk of arithmetic is not. The exceptions leap to mind precisely because, as exceptions, they are memorable.
An obsolete decision, sure, but I wouldn't call it incorrect.
This begs the question, though, why don't we revert it in new architectures, already incompatible with everything. Tradition, I guess?
A) exception/panic on overflow: this is preferable to overflow but also undesirable behavior.
B) arbitrary precision integers: this is nicer, but even today significantly more expensive than ordinary integers.
I don't find this undesirable at all. I have worked in languages where this is the default (Standard ML) and it's quite nice. You can always explicitly use a word-type if you actually want overflow. With hardware support, it is even fairly cheap to implement, as you just connect the overflow bit of your integer ALUs to something interrupt-triggering.
You can exploit the IEEE-754 inexactness exception with doubles if you're okay with 53 bits of integer precision. Tim Sweeney showed me this trick a long time ago, which he combined with run-time code patching so that an arithmetic operation would be speculatively compiled as either fixnum or bignum based on observed run-time behavior. It'd compile it to fixnum by default with an overflow trap using the FP inexact exception. The trap would patch the fixnum site with the equivalent bignum code and resume. With hindsight inspired by modern trace compilers like LuaJIT, the better approach is probably to compile entire traces speculated on either case, so you can hoist most of the guard predicates and not speculate based solely on the proximate call site but on longer traces.
In c, unsigned behavior is also less UB.
And while the enforcement isn't there, it is still better to communicate your intent with the more precise type.
This is correct. However, if you use C, your domain is bits and bytes, not abstract things — because this is what C is created to work with. I love using appropriate abstraction levels for different domains, but if your actual problem has nothing to do with systems programming, you probably shouldn't use C in the first place.
Second, you imply that this would have been an easily solvable problem. If only those that came before you hadn't been so willing to shout "Hooray" and make a choice you disagree with.
I don't think you have to have respect for those that came before you. But blithely ignoring their context when they made these choices is... rude. At best. (It also ignores that many early computers did not use 2's compliment numbers, so it was possible to keep adding to bigger and bigger numbers, you would just run out of space and cause other problems.)
Finally, over/under flow are pretty much endemic into all physical things. Ever accidentally pour too much water into a cup? Why didn't you pick the correct cup for the domain of liquid you were dealing with? Overfill a laundry machine? Run out of ink in the pen you were using? Reach for a sheet of paper to find you had none left? ...
Unsigned integers have a big "cliff" immediately to the left of zero.
Its behavior is not undefined, but it is not well-defined either. For instance, subtracting one from 0U produces the value UINT_MAX. This value is implementation-defined. It's safe in that the machine won't catch on fire, but what good is that if the program isn't prepared to deal with the sudden jump to a large value?
Suppose that x and y are small values, in a small range confined reasonably close to zero. (Say, their decimal representation is at most three or four digits.) And suppose you know that x < y.
If x and y signed, then you know that, for instance, x - 1 < y. If you have an expression like x < y + b in the program, you can happily change it, algebraically to x - b < y if you know that overflow isn't taking place, which you often do if you have assurance that these are smallish values.
If they are unsigned, you cannot do this.
In the absence of overflow, which happens away from zero, signed integers behave like ordinary mathematical integers. Unsigned integers do not.
Check this out: downward counting loop:
for (unsigned i = n - 1; i >= 0; i--)
{ /* Oops! Infinite! */ }
Change to signed, fixed! Hopefully as a result of a compiler warning that the loop guard expression is always true due to the type.Even worse are mixtures of signed and unsigned operands in expressions; luckily, C compilers tend to have reasonably decent warnings about that.
Unsigned integers are a tool. They handle specific jobs. They are not suitable as the reach-for all purpose integer.
There's the rub! You need to justify characterizing your values in that way, which means either explicit range checks and assertions, or otherwise deriving them from something that applies that guarantee in turn. And that justification is more work than just making your code correct for every value. I mean, if x and y are both smallish, is x*y also smallish?
The "nearby cliff" is a good thing, in that it makes errors come out during testing rather than a month after you ship. Handwaving about "reasonably close to zero" is begging for trouble.
In the absence of automatic bigints, unsigned integers are easier to make correct.
For example, Windows has the ScaleWindowExtEx function, which scales a window by some rational number, expressed as the ratio of two ints. Using signed arithmetic is already suspicious: what does it mean to scale a window by a negative fraction? But of course they forgot about the INT_MIN/-1 case, and the result is a BSOD. http://sysmagazine.com/posts/179543/
http://kqueue.org/blog/2012/12/31/idiv-dos/ has some others. Fun stuff.
The question of what bugs you actually see was meant to be a personal one, not one of some dramatic bug you've read about on the internet. The answer of which is the better choice is defined by how much damage is caused by one choice versus the other, and you get that answer by noting how frequently you get bugs in practice as a result of such decisions. (Not that thought experiments and imagination no place, but this is clearly a question for which you can talk yourself into any direction.) For example, I've never had problems with C/C++ undefined behavior of signed integer overflow, while you're spending a lot of time talking about that. I have seen bugs caused by unsigned and signed integer usage that fit into other categories, though.
Personally, the bug I introduce most often with signed ints is a failure to range-check for negative values, e.g.:
void *get(int idx) { assert(idx <= arr.size(); return arr[idx]; }> void *get(int idx) { assert(idx <= arr.size(); return arr[idx]; }
You got problems there even if idx and arr.size() are unsigned.
In order to disallow negatives, you need signed arguments, or else to reduce the range. (Say the numerator and denominator cannot exceed 255 or whatever). Otherwise the function has no way of knowing whether argument values (UINT_MAX-3, UINT_MAX-2) are a mistaken aliasing of -4, -3 or deliberately chosen positive values.
For all we know the ScaleWindowExtEx did have a domain check that disallowed negatives, but it put it after the division.
An defined behavior alternative:
for( size_t i = n; i --> 0; )Edit: Sorry, missed the "i-- > 0" at first. The code works, but not because of changing "unsigned" to "size_t".
It makes sense to use unsigned for sizes to save a bit (or 32 bits per size). Also less invalid possible inputs to handle.
Aaaand you only need to care about this case.
For an unsigned int just check: x < (your max value)
For signed ints: x < (your max value) AND x > 0
Oh, the downward counting loop example. Because that's done very frequently no? I really don't remember when was the last time I did that (I'd much rather have an upwards loop then y = MAX_VALUE - x (adding 1 if needed)
Quite funnily, if you do a loop in x86 assembly, it is naturally downwards counting, if ECX is zero the loop ends
Don't use a for, just i = (n - 1); do { i--;} while (i);
A key difference is whether the type is meant to be an index (counting) or for arithmetic. Indexing with unsigned integers certainly has its pitfalls:
while (idx >= 0) { foo(arr[idx]); idx--; }
But this is outweighed by the enormous and under-appreciated dangers of signed arithmetic!Try writing C functions add, subtract, multiply, and divide, that do anything on overflow except UB. It's trivial with unsigned, but wretched and horrible with signed types. And real software like PostgreSQL gets it wrong, with crashy consequences: http://kqueue.org/blog/2012/12/31/idiv-dos/#sql
Of course that means you need to be aware which calculations are potentially dangerous, you need to know about signed integer overflow problems in the first place, etc. ..
For those curious...
https://www.securecoding.cert.org/confluence/display/c/INT32...
But if you use signed, you have to check for values under zero AND the top of the range. So I don't understand how the bugs were caused by unsigned overflow. Wouldn't they still be bugs with negative results in signed numbers?
By the way, an underflowed/large unsigned number used when a signed number is expected in most contexts (e.g: when adding as an offset) will behave correctly. It will fail when you try to compare it using < or >. If you use enable gcc warnings (which you ought to for every conceivably useful warnings :-) ), that will be caught.
I would like to be clear that I agree with your assessment for most situations, with some exceptions.
For example, if z is unsigned, then both x and y will get converted to unsigned as well. This can cause surprises when you expected the LHS to be negative, but it's not, because the right side is unsigned, and that contaminates the left side.
if (x < y * z) { ... }
This particular case gets caught by -Wall, but there are plenty of cases where unintended unsigned contagion doesn't caught by the compiler. Of course, if you make x long, then y * z will be unsigned, then widened to long, which gives you different results if you are on 32-bit or if you are on Windows. Using signed integers everywhere reduces the cognitive load here, although if you are paranoid, you need to do overflow checking which is going to be a bear and you might want to switch to a language with bigints or checked arithmetic.As another point, in the following statement, the compiler is allowed to assume that the loop terminates:
for (i = 0; i <= N; i++) { ... }
Yes, even if N is INT_MAX. The way I think of it, your use of "signed" means that you are communicating (better yet, promising) to the compiler that you believe overflow will not occur. In these cases, the compiler will usually "do the right thing" when optimizing loops like this, where it sometimes can't do that optimization for unsigned loop variables.So I'm going to disagree as a point of style. Signed arithmetic avoids unintended consequences for comparisons and arithmetic, and enables better loop optimizations by the compiler. In my experience, this is usually correct, and it is rare that I actually want numbers to overflow and do something with them afterwards.
All said, if you don't quite get the subtleties of arithmetic in C (yes, it is subtle) then your C code is fairly likely to have errors, and no style guide is going to be a panacea.
In fact, prior to Java 8, you could not even declare unsigned integers.
In Java 8 you still can't declare unsigned integers. It just that API have been
extended to provide operations that treat an int or a long as having unsigned
value. This seems to imply that Java leaves the behaviour of signed integer overflow
up to the underlying hardware, but guarantees a two's complement
representation even if the architecture does not. It seems reasonable to
expect that it would simply wrap to 0 (I wasn't able to find a more
conclusive reference for this).
In fact Java Specification does specify what happens in case of overflow very
precisely. For example for multiplication (section 15.17.1): If an integer multiplication overflows, then the result is the low-order bits
of the mathematical product as represented in some sufficiently large
two's-complement format.
And for addition (section 15.18.2): If an integer addition overflows, then the result is the low-order bits of the
mathematical sum as represented in some sufficiently large two's-complement
format.
Java approach without undefined behaviour would seem to be quite convenient for
a programmer, but it does not seem to be the case in practice. Of course you
gain ability to check for overflow after performing the operation, which is
nice, but at the same time loose the straightforward way to detect bugs
provided by undefined behaviour (UB).If an application executes UB, then it is obviously wrong, thus a simple instrumentation of arithmetic operation followed by a check for overflow gives a way to detect such problems without false positives (see for example -fsanitize=undefined and similar options available in modern C/C++ compilers).
IMHO only very small fraction of overflow bugs would have been fixed by just wrapping result around.
Using exceptions?
IMHO only very small fraction of overflow bugs would have been fixed by just wrapping result around.
Yes, switching to wrapping integers is not the solution to overflow. Rather check the operation beforehand. I still think using unsigned integers is always preferred if you don't need negative values.
int a = ...;
if (a + 100 < a) overflow
So this is one of cases where wrapping behaviour could potentially fix bugs in
existing applications. Obviously, rewriting this to work with overflow is
trivial if you are already aware of undefined behaviour that could occur.It would be much better to write proper checks or just call functions that do so.
[1] If I was designing a system that could only have one type of int (but why?) I'd use unsigned ints and if I needed to represent negative numbers, I'd use two's complement which is fairly well behaved, much as the author points out. This is fairly common in the embedded world.
Well with that attitude!
char *a, *b;
size_t foo = a - b; /* generates a warning with -Wconversion */
This tells me that C's native integer type is ptrdiff_t, not size_t. (And I agree this is crazy: unsigned modular arithmetic would be fine, but they chose a signed result for pointer subtraction).Why care about this? You should try to get a clean compile with -Wconversion, but also you should avoid adding casts all over the place (they hide potential problems). It's cleaner to wrap all instances of size_t with ptrdiff_t- you can even check for signed overflow in these cases if you are worried about it.
There is another reason: loop index code written to work properly for unsigned will work for signed, but the reverse is not true. This means you have to think about every loop if you intend to do some kind of global conversion to unsigned.
But. Signed means that i < j implies that p + i < p + j, or that p < q implies that p - r < q - r.
($((-2**63/-1)))g << h Well defined because 2147483648 can be represented in a 32 bit unsigned int
Even under those assumptions, it is implementation defined if unsigned int can hold the value 2^31. It is perfectly valid to have an UINT_MAX value of 2^31-1. In that case the code will cause undefined behavior.
The only guarantee for unsigned int is that it's UINT_MAX value is at least 2^16-1, regardless of its bit size, and that it has at least as much value bits as a signed int.
For example C allows these ranges:
int: -2^31 , 2^31-1
unsigned int: 0 , 2^31-1
http://www.pixelbeat.org/programming/gcc/integer_overflow.ht...
The problem with unsigned integers is that most people, past me included, think that it prevents nonsense negative values. It doesn't. The compiler will quite gladly let you call a `void func(unsigned x)` with -1 and let you have "fun" later debugging why who and how this function got called with an enormous value.
And, of course, that's not the only source of negative numbers that get passed in by mistake. More often it's the result of a subtraction that can "never" be negative.
All of the mathematical operations are defined on unsigned int in C. This is not true of int.
Compilers are now starting to do all manner of nasty optimizations on undefined behavior. It is now only a matter of time before an signed int burns you.
Example for counting backwards using unsigned types (underflow check, -1 is 111...111 binary in 2's complement representation, so -1 is equivalent to the biggest unsigned number):
size_t i = ss - 1, j = sso;
for (; i != (size_t)-1; i--) {
switch (s[i]) {
case '"': j -= 6; s_memcpy6(o + j, """); continue;
case '&': j -= 5; s_memcpy5(o + j, "&"); continue;
case '\'': j -= 6; s_memcpy6(o + j, "'"); continue;
case '<': j -= 4; s_memcpy4(o + j, "<"); continue;
case '>': j -= 4; s_memcpy4(o + j, ">"); continue;
default: o[--j] = s[i]; continue;
}
}
Example for compute percentage on unsigned integers (same idea can be applied for signed) without losing precision nor requiring bigger data container (overflow checks): size_t s_size_t_pct(const size_t q, const size_t pct)
{
return q > 10000 ? (q / 100) * pct :
(q * pct) / 100;
}
Also, for cases where overflow could happen, you can handle it doing something like this: sbool_t s_size_t_overflow(const size_t off, const size_t inc)
{
return inc > (S_SIZET_MAX - off) ? S_TRUE : S_FALSE;
}
size_t s_size_t_add(size_t a, size_t b, size_t val_if_saturated)
{
return s_size_t_overflow(a, b) ? val_if_saturated : a + b;
}
size_t s_size_t_mul(size_t a, size_t b, size_t val_if_saturated)
{
RETURN_IF(b == 0, 0);
return a > SIZE_MAX / b ? val_if_saturated : a * b;
}
P.S. examples taken from:https://github.com/faragon/libsrt/blob/master/src/senc.c (underflow example)
https://github.com/faragon/libsrt/blob/master/src/scommon.h (overflow examples)
The idiomatic way is
for (size_t i = ss, j = sso; i-- > 0; ) {
...
}