Hacker News Reader

Comment by _RPM | Hacker News Reader

It's _very_ important to know if it failed if you don't want to introduce undefined behavior. With undefined behavior, _anything_ can happen, (e.g. arbitrary code execution). But let me make it clear: malloc failing doesn't mean that undefined behavior will happen. However, pretending like malloc didn't fail when it really did will most likely get you undefined behavior. You're not going to segfault either way if you handle things correctly. Consider a web server running out of memory to allocate a request structure, do you think it is a good idea to just let it SIGSEV?

View on HN

I have some questions:

Have you experienced malloc failing a lot? How do you test to ensure error handling code paths work as expected?

Have you read 'notes' section of Linux malloc manpage? It says:

By default, Linux follows an optimistic memory allocation strategy. This means that when malloc() returns non-NULL there is no guarantee that the memory really is available. In case it turns out that the system is out of memory, one or more processes will be killed by the OOM killer.

I have never knowingly experienced malloc failing in any of my code which is why I asked if it is something to worry about in practice. Once you run out of memory things start dying or the system locks up hard.

Edit. It is impossible for malloc to fail (at least on linux) [1]. Seems rather pointless checking for something that can’t happen.

1. https://scvalex.net/posts/6/

That link shows that malloc failed, just not due to exceeding available RAM. At the end it says "malloc finally failed because of address space exhaustion."

If the memory was actually being used it would have crashed long before that. The test was kind of artificial.

Absolutely. I was pointing out that "It is impossible for malloc to fail" is, according to the linked page, an incorrect summary.

It is still pointless checking if malloc fails on linux :)

Actually the real issue here is that you should be aware that on some systems malloc might fail and not segfault.

On virtual memory systems, it is a safe bet that malloc will "never" fail to return a value. It is just good form to check the return value of system calls. If you program for embedded systems (e.g. Arduino) and bring over the bad habit of never checking malloc, you are going to be in for a surprise. I'm not sure, but I believe on iOS, malloc could fail on the device.

I had malloc fail recently, on a machine with low rlimits. Here's an example on FreeBSD 8.2:

    #include <stdlib.h>
    #include <stdio.h>
    #include <sys/resource.h>
    
    const int SIZE = 64*1024*1024;
    
    int main() {
      char *buffer = NULL;
      struct rlimit r;
    
      /* Limit all memory to 1024 bytes */
      r.rlim_cur = 1024;
      r.rlim_max = 1024;
    
      if (setrlimit(RLIMIT_AS, &r) == -1 ) {
        perror("setrlimit(RLIMIT_AS) error\n");
        exit(EXIT_FAILURE);
      }
    
      if ( (buffer = malloc( sizeof(long) * SIZE) ) == NULL) {
        perror("malloc() failed\n");
        exit(EXIT_FAILURE);
      }
      free(buffer);
      printf("Malloc worked\n");
      return 0;
    }

Here's the output:

    %cc fail.c
    %./a.out
    malloc() failed
    : Cannot allocate memory

If malloc failure testing is important to you, then you'll need some way to replace your use of the system malloc/free with special versions of your own. (Eg, through function pointers, or LD_PRELOAD.) Your own versions can be crafted to insert faults in just the right spots.

It's tedious work.

If you have run out of memory there is not much you can really do except crash. Even logging is a problem. Better to just crash and deal with it.

I was under the impression that compiler optimisation often removed any NULL checks anyway [1].

1. http://blog.llvm.org/2011/05/what-every-c-programmer-should-...

> I was under the impression that compiler optimisation often removed any NULL checks anyway

Goodness.

No, you are misunderstanding the example. The optimizer did not introduce a bug by removing a NULL check. The code had a bug because it dereferenced the pointer before checking for NULL.

Yes, but they were making the point that thinking you are checking for a null pointer may not always hold especially once inlining happens.

Are there any modern platforms where dereferncing a NULL pointer will not cause a crash?

Edit. Answering my own question it looks like ARM is one [1].

1. https://www.securecoding.cert.org/confluence/display/c/EXP34...

> thinking you are checking for a null pointer may not always hold especially once inlining happens

Incorrect. The example code did not check for a null pointer. It dereferenced a pointer, which implies that the pointer will never be null, and then it pointlessly checked for null on something that could not have been null. The optimizer in fact made the code more clear and easier to read, while maintaining correctness.

Quoting from the reference:

"While this is intentionally a simple and contrived example, this sort of thing happens all the time with inlining: inlining a function often exposes a number of secondary optimization opportunities. This means that if the optimizer decides to inline a function, a variety of local optimizations can kick in, which change the behavior of the code. This is both perfectly valid according to the standard, and important for performance in practice.”

In practice derefencing a NULL pointer on any x86 platform will always cause a crash (defined undefined behaviour I guess), but you can’t assume this on other platforms (ARM seems to be the big one).

I have to say before today I had assumed dereferncing a NULL pointer would always cause a segfault. It is always good to learn something new.

You can't be further from the truth. A vast magntitude of bugs are due to the system running as if nothing has happened operating on garbage data, and giving you garbage results. If your system is robust to garbage data, say neural network training, good luck finding the bug then.

But what do you do in practice if you have run out of memory?

One approach is to free memory, retry, and hope for the best.

e.g. https://github.com/git/git/blob/master/wrapper.c#L63

Another is to pre-allocate enough reserve memory to be able to save and exit cleanly. When malloc fails, switch to the reserve memory and either shut down nicely, or post a dire warning to the user that the end is neigh and to "SAVE AND EXIT IMMEDIATELY".

Reply on news.ycombinator.com