Suppose we're on a 32 bit platform and num is 0xF0000001. When multiplied by 64 (0x40), we'll end up allocating a 64 (0x40) byte buffer. But other code converting num to unsigned may be expecting a buffer large enough for 0xF0000001 64 byte records. After all, that was probably the reason for multiplying by 64.
Passing a large value into the method shown in the article will do nothing nefarious (assuming sizeof(size_t) >= sizeof(int)). It will either return a large allocation, or, more likely, fail because the amount of requested memory is too large.
If you have a narrowing/casting bug somewhere else in your program, which BTW would produce an obvious warning, that would of course cause trouble as you have described.
And while mixing signed and unsigned arithmetic for buffer sizes is, of course, a recipe for desaster, I think it's incorrect to claim that the allocatebufs method shown in TFA has a "backdoor" because of this. I feel that is a bit like saying memcpy has a backdoor because you might get your pointer arithmetic wrong when calling it.
printf("%p\n", malloc(0x7fffffffffffffff));
prints (nil)
on my amd64 machine, as there is no way for the kernel to allocate that much virtual address space. (Even if it wouldn't back it w/ physical RAM.)The scenario being discussed in this subthread — having a negative number get inadvertently converted into a `size_t` and passed to `malloc()` is exactly the sort of way you end up getting a NULL back. But this is also not guaranteed.
TFA calls this a "backdoor"; so how do you actually "get in" after you managed to get the backdoor through code review and deployed into production?
For example, suppose the code was parsing user input as follows (a fairly common pattern):
unsigned int count = read_int();
struct buf *bufs = allocatebufs(count);
if(!bufs) goto fail;
for(unsigned int i=0; i<count; i++) {
bufs[i] = read_buf();
if(!bufs[i]) break;
}
This code isn't really safe because it passes an unsigned int to allocatebufs, but by default you won't see a warning for this. In the previous version of the code it would work fine - reject anything above 256. In the new "fixed" code, if count = 0x20000001 (for example) this will allocate 64 bytes and proceed to read up to 34 GB of data into the buffer. (A clever attacker can probably cause read_buf to fail early to avoid running off the end of the heap).That code doesn't need to be buggy - simple, correct code that traverses e.g. a linked list can be abused to gain arbitrary memory writes and reads if you can overwrite the pointers in that linked list with values under your control; and these arbitrary memory writes can be abused to gain arbitrary code execution.
Exploiting a heap overflow is not as straightforward as a stack overflow, but certainly possible, there are many real world code execution vulnerabilities that arise from a buffer overflow in heap.
If you ask the method to allocate a negative number of bytes and the method returns a buffer which is greater than zero, that doesn't seem like a backdoor in the allocation method!
Saying you can get a return buffer that is smaller than whatever amount of bytes you requested is wrong I think. Can you give an input to the second method that will result in a buffer smaller than the input value?