Malloc Never Fails (2012) | Hacker News Reader

Malloc Never Fails (2012)

scvalex.net

125 pointsKirinDave156 comments

The title here is just blatantly false; there are certainly some scenarios in which it won't fail, but many in which it will. The author is aware of this since he fixes the claim toward the end:

> To clarify, the surprising behaviour malloc has does not mean we should ignore its return value. We just need to be careful because malloc returning successfully does not always mean that we can use the requested memory.

It's worth pointing out that if you turn overcommit off (https://www.kernel.org/doc/Documentation/vm/overcommit-accou...) you will, in fact, get this guarantee

> does not always mean

> if you turn overcommit off

The author said ‘does not always mean’ so giving one condition in which it does doesn’t prove them wrong does it?

You're right, based on the context of the post, I was interpreting that statement to make a broader claim than it was; I've modified my comment not to contradict it.

It's a shame that turning off overcommit is done at the system level, rather than the malloc call level. Some applications, and perhaps some allocations within otherwise naive applications, might prefer to have an allocation fail early, where failure could be sensibly handled.

I think that would ultimately have undesirable consequences.

If this were an option on a malloc call, any such call would have to reserve the allocated memory at that point, reducing the usefulness of overcommit for other processes. This would set up a 'tragedy of the commons' scenario, where every application developer defensively uses this feature because other applications are using it.

It's already quite easy to implement manually though, just do a write after allocation.

That's a good point, though I think one would have to write to each of the pages allocated to reserve the full extent of the memory allocated by the malloc call.

You could mitigate that by treating these allocations conservatively: they only succeed if they can reserve the memory they want and there is 'enough' uncommitted memory left to cover all as-yet-uncommitted normal allocations. So, it would fail long before a normal allocation would.

I don't know quite what 'enough' means, but this whole business is already a rat's nest of heuristics, so one more should fit in nicely.

There is some support for this actually. Namely, when overcommit_memory is set to its default value (0), then it actually implements a heuristic for overcommit. (Where as the value `1` corresponds to "always overcommit.") Namely, only when overcommit_memory is set to 0, if you allocate memory with mmap and pass the MAP_NORESERVE option, then allocation behaves as if overcommit is always enabled and there is no check. But if you don't pass that option, then there is a heuristic check which could cause malloc to fail.

I discovered this because of differences in how jemalloc and my system allocator (from glibc) allocated memory. Namely, jemalloc would pass the MAP_NORESERVE option, which meant my program behaved as if overcommit was always enabled. Thus, an out-of-memory condition meant that it was subject to the OOM killer. But when I used my system's allocator, the MAP_NORESERVE option was not passed, which led to the heuristic detecting an out-of-memory condition and causing the malloc call to fail.

For more info, see `man 5 proc` and/or my exploration here: https://github.com/BurntSushi/ripgrep/issues/993#issuecommen...

(It's all still a bit wishy washy, so I think your point still stands. But I figured I'd chime in with some bits that don't appear to be widely known, and which led to some surprising differences in behavior based on which allocator one chooses.)

The OOM killer can strike at any time, regardless of the mapping options you've set, even if you aren't explicitly or implicitly allocating new memory. What I'd like, and what I think twic is asking for, is a way to "opt-out" of the overcommit paradigm at a process level, so that allocations by a given process reserve memory and may fail, but in return the kernel promises not to OOM kill it.

Most of my programming experience is in Windows. Can someone briefly explain why it doesn't seem to need an OOM killer? Is it happier to page? Or it always commits on allocation? Or something else.

Windows does not have fork(), which as a sibling comment indicates is one of the main issues with removing overcommit.

AFAIK windows lets whatever application asks for memory and can't get it crash, whereas the OOM killer tries to be intelligent about what gets killed.

AFAIK, Windows always commits on allocation, growing the swap file if necessary.

Why doesn't Linux do this?

>so that allocations by a given process reserve memory and may fail, but in return the kernel promises not to OOM kill it

The issue with this is that there are other processes on the system. If I start a program that used your idea for allocation, and it used 90% of my memory, a small growth in my memory usage might mean I run out. But because it's guaranteed not to OOM, another program gets killed rather than the one using 90% of the memory. If you want no OOM kills, you've got to disable overcommit globally.

n.b. turning over-commit off comes with its own set of problems, such as causing programs to fail long before all memory has really been exhausted.

For example, fork() will have to ensure that there is enough memory for a complete copy of the running process. If you have a large process, e.g. using 4GB of a 8GB machine, then fork() won't be able to run, even if you just want to fork and run a tiny program.

With over-commit turned on, the fork() would work (because of copy-on-write) and the program could then happily exec() the new program.

The work-around is to allocate huge amounts of swap space so that the OS can be confident that it can reserve all the potentially required memory from fork(), even if it never normally has to use all that memory.

Or if you're just forking to exec a tiny program, use vfork().

cow is orthogonal to overcommit

Redis is an example of this. When redis saves in the background, it will for and use COW to shapshot a point in time, while the main redis thread keeps going with the normal redis. If you disable overcommit, redis can fail to fork, when there is actually sufficient memory to save the db.

> It's worth pointing out that if you turn overcommit off ... you will, in fact, get this guarantee

But many Linux applications assume the overcommit behavior. Its far easier to just "go with the flow" or "When in Rome...". Overcommit is the programming culture of Linux and should be assumed when writing Linux apps.

That has not been my experience. I disable overcommit on my machines and it has been considerably nicer to have apps exit due to malloc failing than random processes getting OOM-killed.

My experience with the oom-killer has been consistently terrible. Stuff like the X server getting killed taking down the whole user session instead of whatever was actually using the most memory. Probably happening because it's the parent process to everything else, but it's still a mind-blowingly dumb way to handle the low memory condition.

> The title here is just blatantly false

intentionally. you don't need to point out the "error". it's a rhetorical device.

The malloc(3) family will absolutely fail on OpenBSD, for many reasons: login.conf/ulimits, calloc(3) for when integer overflow is detected (nmemb * size), same for OpenBSD's reallocarray extension. And yes, because the system was unable to satisfy the requested allocation.

Linux overcommit, and developer mindset is a detriment to software quality and portability.

That's a needlessly confrontational statement. Yet I find myself in agreement. Sigh.

This assumption lead to Rust's standard library not having a way to catch allocation failures (which is only now being rectified, and only partially).

It's very Linux-centric and presumes a certain config+usage pattern. Not true on Windows. Not quite true on macOS. Not true in WASM. Definitely not true on embedded platforms.

Is there somewhere to read about how rust is tackling this problem?

It seems to be described in this Rust RFC: https://github.com/rust-lang/rfcs/blob/master/text/2116-allo...

A short overview:

1. Rust the language knows nothing about allocation. If you care about this behavior, it mostly limits the code of others' that you can use, but you can always write your own versions of things that respect fallible allocations.

2. Rust's standard library assumes memory is infallible. This is partially because it's a good default, and partially because our allocator API was not ready yet.

3. We've been working on the allocator API.

4. We have a rough plan for parameterizing data structures over allocators.

5. If this topic is of interest to you, https://github.com/rust-lang/wg-allocators is where to get involved.

Attempting to catch allocation failures is the stupidest feature of C++ and the main reason that people turn off exceptions in that language.

Malloc allocates virtual memory. The original article had to issue a correction at the end: "I was wrong about why malloc finally failed! @GodmarBack observes, in the comments, that x64 systems only have an address space of 48 bits, which comes out to about 131000 GB. So, on my machine at least, the malloc finally failed because of address space exhaustion."

That's incorrect, 2^48 bits = 262144GB actually.

But x86-64 divides the 48-bit address space into two halves, only one of which is available in the user mode.

To be fair, there's nothing that says the higher half _has_ to be entirely kernel mode. Only that bits 63 through 48 have to be the same value.

And Intel has a spec out for a PML5 page table, giving you 57 total virtual address bits.

They forgot to mention the kernel gets the top half of the address space, leaving 2^47 for the process. And then it mostly adds up.

What bothers me is that I read this train wreck without any red flags until I saw his correction at the end. Even the headline was wrong given the article, which itself was wrong.

I really should have coffee before HN

What exactly is "wrong" about the article?

Linux's Overcommit behavior is non-obvious to many programmers. Its one of those issues that very few programmers I've come across in the workplace understand properly.

This blogpost properly understands the issues associated with Overcommit, and have done some preliminary investigations that describe the behavior. Its a really good blogpost.

The general point of the blogpost is that "Malloc Fails due to address space exhaustion more often than actual memory-exhaustion". Because actual memory exhaustion causes OOM killer code to be run... and OOM killer is a non-obvious case of the Linux kernel.

What's "wrong" about the article is that the author has no idea what malloc(3) does, or about the wide range of execution environments C programs find themselves in and the different ways that the API maps to those environments.

More generally, the article does nothing to educate the public or move the "debate" about overcommit in any sort of useful direction.

What's wrong with the article is that malloc is not even a feature of Linux. There's mmap and brk, both of which have documented failure modes.

Most typical programmers will be using malloc, or some mechanism built on top of malloc (C++ new). MMap and brk are useful for certain situations (explicitly getting Huge Pages), but I don't think that the typical programmer necessarily needs to know about those.

Since glibc malloc is built on top of mmap and brk, I think your distinction is mostly academic. For any programmer using Linux and glibc... malloc's failure mode IS mmap and brk failure modes.

It is a mischaracterization to say that C++ operator new is built on top of malloc. If your new is overridden by, say, tcmalloc, your program will never call malloc.

It is a feature of your chosen libc, which has to then interact with those system calls, in a way that gives malloc a peculiar behavior on Linux specifically as opposed to other OS.

Interesting topic with many things to say about, but wrong content. The point is that in most C programs, it is not worth to handle OOM errors, because what you can do during OOM is of very little value, on the other hand handling OOM correctly is very hard. However you can't do this in libraries, because you don't know how the library is going to be used. So for instance in order to make my Radix tree library resistant to OOM failures, I had to write a specific fuzzy test that used a malloc failing with a given probability, and check if the tree is sane after some stress-work with such malloc. In general the complexity of handling OOM in complex programs that deal with complex data structures is not often recognized.

When writing C libraries, it's good form to be able to init your instance with allocation and logging function pointers. That lets you play nice with most any env you're being pulled into, and gives your consumers an obvious place for nice hooks for debugging.

Yep, that's what I do in order to be able to fuzz test with an OOM returning malloc() in my lib. Agreed on the fact it's good form.

Another thing to keep in mind, though 32 bit is less and less common over time, malloc would probably fail on a 32 bit process that is out of address space.

Raspberry Pis almost always run 32 bit OSes, so that's a common place to encounter 32-bit.

I had always thought they'd be running a 64-bit OS, given that the new hardware is 64-bit. TIL that Raspbian is still 32-bit!

It seems Broadcom never released the required binary blobs in 64-bit format. People love to hate on Intel, but the ARM SoC world is quite terrible about binary blobs.

Unless you need the address space, there is little gain with a 64 bit OS. The wider pointers use more RAM and eat up more of the available memory bandwidth and caches. So not swotching to 64 bits is likely the best use if the hardware.

It’s a toss up, I think: actual results depend on the task at hand. 64-bit has the benefit of more, wider registers, which can speed up certain tasks.

I stand corrected. I missed the fact that ARM also introduced new registers in AArch64. I thought that such a change only happened in amd64.

The new registers in AArch64 aren't as useful as on AMD64 because ARM wasn't as register-starved as x86 to begin with.

The other thing about amd64 is that amd64 code can safely ditch old features like x87 floating point, which is much slower than more recent features. Also, function calls will more frequently pass by register. And I can think of one OS specific feature that is better on amd64: when MS ported to amd64 they took it as an opportunity to speed up the ABI for Structured Exception Handling (SEH). The x86 one incurred a time cost even for code that did not throw.

Of course none of this applies to arm/aarch64.

I saw the title and thought "oh heck this isn't right at all"...

Set VM Overcommit to zero on an embedded system with no swap (a Raspberry Pi will do nicely).

Write a C program that malloc()'s all the RAM.

Watch malloc start to fail when you hit the RAM limit and the kernel has dumped all the I/O cache it can.

I think the author uses a hyperbolistic title, but if you read the article it s addressed that there are ways it can fail.

The larger point is that very few users of malloc understand its semantics, and in fact you can't know exactly how malloc will behave without knowing things about the runtime configuration of the system (as opposed to the hardware availability as many people like to think the simple case is).

When I use Python sometimes I run into a MemoryError when working with large datasets. How does the Python runtime know I am out of memory if the kernel won't tell it. Does it try a write and catch the signal?

Perhaps any write can fail, and if one does then it triggers some code which frees some pre-reserved space which allows the interpreter to continue, and properly unwind the stack.

I believe that the python interpreter creates a MemoryError at startup, and throws that when you run out of memory.

Ok, but while unwinding a try/except block, the interpreter might need some extra memory, I suppose, depending on the code in the except clause.

If your large dataset leads to a single huge memory allocation, the kernel will fail the allocation even though overcommit is enabled. Early versions of libvpx had a similar issue, where it allocated a huge amount of memory at once (actually, told the dynamic linker to do so) which failed on low-memory machines (https://bugzilla.redhat.com/show_bug.cgi?id=600663 "libvpx.so.0 fails to load on low-memory machines due to ridiculously oversized .bss").

malloc fails with ulimit

Very relevant, because containers are often under what basically is ulimit.

Not that you can do much when malloc fails...

"graceful degradation" is one of the things worth trying.

> Not that you can do much when malloc fails...

Tell the user they can’t do that operation? Use on-disk storage instead? Re-use memory you already have (such as evicting a cache and taking the memory it already had allocated)? Abandon what you were doing if it was only an optimisation and wasn’t essential (such as allocating memory as part of a speculative execution)? Run a garbage collector and try again?

Lots of options available in some situations.

In practice i haven't seen any application to gracefully handle out of memory situations. Even back in Windows 3.x days when running out of memory was common, most applications simply hanged or crashed (which also took the entire GUI with them) and those that didn't often had a "no memory, kthxbye" dialog that shut down the application. A very few rare cases would try to display some "sorry, no memory, close some programs and try again" dialog but even those were hit and miss, depending on the situation.

Once 32bit hardware-based virtual memory entered the picture, every single application would just assume you have endless RAM - memory checks are reserved for "common cases" like trying to create a 100000x100000 image in a (32bit) image editor.

The reason for all that is simple: out of memory situations are stupidly and increasingly rare and in most cases where they can happen there isn't much you can do (e.g. what would you do if you run out of memory while making the fourth button in a toolbar?) and really in the 99% of the cases there might only be five people in the entire universe that will encounter such a case (two of them will tweet about it though and amass a lot of "lol, those garbage developers" retweets) so littering your codebase to keep those five people (and their Twitter followers) happy is not worth the effort. I mean, are you really going to put a "run out of memory" check after every toolbar button allocation? And what are you going to do if that fails? What can you do?

AFAIK some modern languages nowadays even assume memory allocations wont fail (and if they do they just terminate).

> what would you do if you run out of memory while making the fourth button in a toolbar?

Rollback what I’d created so far, evict caches, ask malloc to trim, try again, if that failed roll back again and ask the user to reduce the volume of application data open by closing views or documents or whatever before they try again.

But yeah it’s a lot of engineering.

So you meant to say: ignore the possibility of this happening because it is extremely unlikely, too complex to handle and your time is better spent fixing much ore serious bugs that the users report.

Silly engineering like what you suggest is a waste of time. Non-critical applications that are allowed to crash (every normal desktop app) would end up overengineered and unmaintainable. Embedded systems that must give guarantees about their reliability simplify system design drastically. One of the things going out the window first is dynamic memory allocation. It has too many failure modes.

> ask the user

You just failed to create a toolbar button, how are you going to ask a user do something if you already failed to create a tiny UI element?

I guess you pre-allocate these UI elements when you start your application.

What if malloc fails at that point because some other application (we're in a preemptive multitasking OS) decided to gobble up all RAM?

If your application fails at start-up, you don't need to worry about saving state and graceful failure.

Keep in mind that Windows 3.x wasn't really the pinnacle of computing in its days. PCs were relatively irrelevant compared to the systems everyone apart from a mom and pop shop ran their business on.

So the typical behavior you saw was a combined effect of physical constraints and lack of skill or care.

No it wasn't which is exactly why i brought it up (well that, and it was the most popular - despite its shortcomings).

Plus, you can do all the graceful malloc() failure handling that you want, but if the kernel hasn't got overcommit disabled, then your program can still exit uncleanly when you later on try to use some malloc()d memory that was overcomitted.

Exiting cleanly would mean trapping SIGBUS/SIGSEGV (can't remember which one is invoked) and being able to roll-back from just about any point in your code or libraries)

You need to ensure overcommit is disabled, but because that's a system-wide setting, your program can't easily force that on.

https://www.sqlite.org/malloc.html is a welcome contrast to the typical behavior you're describing, but I agree that such measures are rare. (and, not an application: though it provides the mechanisms for applications to follow its lead.)

> In practice i haven't seen any application to gracefully handle out of memory situations

I have seen code hit an allocation failure, bubble the error up the stack, which causes various pieces of code along the stack to free their heap allocations, all the while the process is able to log what is going and keep running.

I have even seen this happen when the allocation that failed was tiny, less than 100 bytes.

What is your definition of graceful in this case? Presumably there was a “business need” for the allocation and the software isn’t just allocating for fun. An allocation failure is a hard fault, it’s not possible to honor the business need so something expected to happen cannot happen. The program could segfault when the null pointer returned is written to, that is kind of crappy. It could report the error “hey we are out of memory and couldn’t do xyz” or better “hey we are out of memory and couldn’t do xyz, this is probably because of abc, maybe you can adjust the tuning.” Or it could be a completely temporal thing, maybe you flush some caches or something to free up some memory and reschedule the work.

I don’t know, maybe this is old school, but yes you should put a check on every toolbar allocation for two reason: one you can gracefully report why your app isn’t doing it’s job and two it is a very very slippery slope when you start ignoring those errors as a matter of practice

> It could report the error “hey we are out of memory and couldn’t do xyz” or better “hey we are out of memory and couldn’t do xyz, this is probably because of abc, maybe you can adjust the tuning.”

What if you cannot do that report because the reporting itself needs memory that can fail?

> but yes you should put a check on every toolbar allocation for two reason: one you can gracefully report why your app isn’t doing it’s job

This is a reason to put checks in every toolbar allocation, but on the other hand there is also a reason to not do that: you are trying to catch a case that will only happen at a 0.00001% of the time yet to do that you are making the codebase more complex which will affect working with it 100% of the time.

Unless you are working in something like a medical or nuclear device, it is not worth to bother with such things. Even then when it comes to high risk programs relying on the programmer doing things right outside the core functionality is probably (i haven't worked on such projects myself so i'm guessing here) not a good idea and instead you should compartmentalize (e.g. the core functionality is running as a separate process from the UI and if the UI crashes, a watchdog restarts the UI process). But that is my guess here, i mainly have common end user/consumer applications in mind.

> Unless you are working in something like a medical or nuclear device, it is not worth to bother with such things.

Where "something like a medical or nuclear device" implies a piece of software that is expected to work correctly.

In other words, you should always ensure that failures resulting from malloc(3) returning an error (and all other errors) are suitably contained.

No, i did not implied that, please do not put words in my mouth.

My implication was software that if it fails it will kill people.

Otherwise if a program crashes because of a situation happens once every 100000000 runs, not only is worthless to worry about it, it actually is preferable to not do that as to keep the codebase clean and hence easier to maintain for bugs that actually do affect people.

Keeping the codebase clean is why I prefer that allocation errors throw an exception rather than returning NULL.

But then you have to worry about exception safety on every code which might call an allocating function, making the code more complex. This is worse on C++ where even innocent-looking code could be allocating memory in the middle of an operation (for instance, by calling a copy constructor which allocates).

> What if you cannot do that report because the reporting itself needs memory that can fail?

Allocate and reserve error-reporting memory early. For example, your ErrorReporter instance is initialized at start-up, and doesn't call malloc() when you make a report.

> What if you cannot do that report because the reporting itself needs memory that can fail?

Allocate a big block of memory at startup. Free it just before doing the reporting.

This is true, in my experience. If you keep your program fairly simple, implement it in plain C, avoid recursion, and keep these requirements in mind from the start, then it is possible to write a program that fails gracefully when it runs out of memory. Some old-school Unix daemons do this. But for most software it's impractical, so you might as well define your own xmalloc() that calls abort() if malloc() returns zero and at least have an immediate and obvious failure, rather than obscure memory leaks and data corruption.

An exception-safe C++ program can do the same through a std::bad_alloc exception.

If the out-of-memory was caused by something simple like accidentally loading 2G of data because of some odd data in an network request or a user selected file, and it rolls back to the start of the UI action or the incoming network request, the application may still work fine.

(I usually connected other out-of-resource conditions such as pipe/socket/open failures to std::bad_alloc too. They're pretty comparable in effect and give a bigger chance of actually exercising those exception paths)

> If the out-of-memory was caused by something simple like accidentally loading 2G of data because of some odd data in an network request or a user selected file, and it rolls back to the start of the UI action or the incoming network request, the application may still work fine.

If you have a system that handles independent requests, it might be more robust to model each request as its own process, and then simply allow those processes to crash if they find themselves in an unexpected condition (and handle process crashes).

It's certainly more robust, but a lot of overhead for what should be a very uncommon situation.

In eg an application server, requests themselves may be independent, but still share a lot of cached data - and as long as locking overhead doesn't overwhelm you, a single process is still the fastest way to share data between (worker) threads.

If that data is read-only then a fork()ed child process will share it just as efficiently as a thread would.

If your threads need to share interleaved modifications to some common state then that approach isn't viable. But in that case it will be very hard to "roll back"/"skip over" a failed request, as it's difficult to be confident that you haven't corrupted that shared data in a way that will cause the same problem for future requests.

fork only works if the central cache never updates. Eg a cache of compiled bytecode for scripts gets filled as those scripts are requested. A cache of database results fills as those results are requested.

I'm not sure why skipping over failed requests is hard. VMs associated with the request are aborted, the client gets a 500/503 and can retry later. Incomplete data doesn't get added to the caches at all, so no corruption of shared data.

It kind of goes both ways. If the data sharing pattern is simple then it's still quite simple to handle that at an inter-process level (e.g. perhaps you have to do cache updates via slower IPC, but if the cache updates only happen after the request has already been handled then they're not on the critical path. Or you might have the parent process handle the cache updates itself). If the data sharing pattern is complex and interleaved then it's hard to achieve with anything other than threads - but that's exactly the case that also makes it hard to avoid corrupting shared data.

You can use pool of processes, so you don't have to launch new process for every request. Also you can and should use shared memory for shared data. In the end it's almost like threads, but with more safety.

Unfortunately, that only works well in the absence of threads. With multiple threads, the out-of-memory could have been caused by code in another thread allocating too much.

If you write code with memory exhaustion in mind, which mostly means not relying on allocations in the error handling paths, malloc fails are perfectly fine to handle.

If you take a look at the Linux kernel, a hugely complex and sophisticated piece of software, you'll find that it gracefully handles allocation failures.

The kernel's "graceful handling" of allocation failures includes killing unrelated processes in an attempt to free some memory.

Also just deadlocking. The kernel's behavior in the face of real memory pressure is actually quite poor. It's unsafe to run a Linux box out of memory.

Depends; for example one video application I've written buffers as many incoming frames in memory as possible; if malloc fails when taking the buffer from 8000 to 9000 frames it's not a problem; just carry on running with the current buffer size.

You still have to gracefully shut down the application by closing file handles, aborting database transactions, etc.

The OS will close the file handles and any database worth its salt should also abort the transaction automatically when the connection to it is lost before the transaction is finished.

When a memory control group runs out of space the process allocating just gets unilaterally killed with no notification. ulimit instead makes brk or mmap return a failure code, which the application could attempt to handle.

How does one guarantee that allocated memory is real? Can you easily wrap malloc to zero/poke the memory in a way that catches all exceptions and guarantees yay or nay ?

Jens Gustedt suggested the following:

    memset(malloc(size), 0, 1)

The program will crash if malloc returns NULL or the memory is not writable.

And then memory deduplication comes along, finds out a huge number of identical pages (all zero) and decides to deduplicate them. So if you want your memory to stay real, I suppose you'd better fill it with random data or something.

You must use mlock call if you want to guarantee that no later action can suffer a major page fault.

You can arrange for malloc to fail. If overcommit is disabled with the right sysctl parameters, then mmap will fail if there isn't enough physical memory to materialize the entire mapping. Even under overcommit, very large malloc requests that translate directly to large mmap requests can fail with a null return.

You can disable memory overcommit in the kernel, before any of you start assuming you can drop your malloc checks.

What happens if you really run out of physical memory (including swap) after overcommitting? Does your process get a signal, or will OOM killer just run without notifying the process that triggered the condition?

Ig you try to allocate more than the system is willing to overcommit (e.g. a huge block all at once), malloc will fail. But if phyaical memory gets exhausted by accessing previously allocated pages, the OOM killer will evebtuslly come around and kill processes without signalling. Signal handlers could still make thenprocess (unknowingly) request more memory, so there is 0 guarantee that a handler could even run successfully.

Oh yeah, I didn't think of that. I wonder if you could write a signal handler carefully to not allocate any memory, stack or otherwise, or is some return address or an internal structure being allocated transparently...

Just trying to allocate stack space for the signal handler may cause the stack to spill into a new page. That is absolutely out of anybody's control. And if that new page cannot be provided, it's game over.

Makes sense. I was thinking that maybe signal handlers could use the regular stack of the thread, but that would of course make everything fall down if the "real" code would write to the stack before updating the stack pointer.

> Section III of this Phrack article is a down to earth description of how the glibc malloc implementation works, if you’re curious.

This is probably outdated, given that it was written in 2009.

pocorgtfo 0x18 contains a more recent glibc malloc exploit which goes into quite a bit of detail if you're interested.

And then, some day, your code gets compiled for Windows under mingw/msys.

I always assume malloc success and that if it were to return null the system is screwed anyway and the app will just abort. I write a lot of 'critical' C/C++ now on embedded systems with mere KB of RAM and just never use the heap ever.

malloc fails with a negative argument

malloc takes a size_t, which is unsigned. A "negative" argument is just something that's absurdly large.

On 64 bit systems it would result in a call to malloc asking for between 9.2 and 18.4 Exabytes.

Translation: This is a euphemism for saying Linux blatantly violates the language specification with no remorse.

(Yes, I realize Linux is the kernel, etc.)

Which part of the specification does this violate?

Like, every part? It doesn't actually allocate the space requested, it doesn't indicate failure by returning NULL (or by any other reliable means for that matter)...

I had a look, and the C standard doesn't actually say anything about indicating failure. Here is all it says about malloc:

> The malloc function allocates space for an object whose size is specified by size and whose value is indeterminate.

> The malloc function returns either a null pointer or a pointer to the allocated space.

I think it's certainly debatable whether overcommitting while lazily allocating counts as actually allocating.

> I had a look, and the C standard doesn't actually say anything about indicating failure.

> The malloc function returns either a null pointer or a pointer to the allocated space.

Uhm, either malloc returns a pointer to the allocated space, or it returns null. I don't see what's so complicated about this. There's no provision for "return a non-null pointer to unallocated space".

The malloc() function requests memory from the kernel using mmap(). If mmap() returns successfully, it means that the kernel is saying that the memory is allocated. So from malloc()'s perspective the memory allocation has been successful and it must return a non-null pointer to the caller. The standard does not require malloc() to distrust what the kernel said and try to actually write to that memory to double-check if there is any physical memory mapped to it or not. The standard also does not impose anything on the kernel. I see no section of the standard being violated here.

Section 7.22.3.1:

The order and contiguity of storage allocated by successive calls to the aligned_alloc, calloc, malloc, and realloc functions is unspecified. The pointer returned if the allocation succeeds is suitably aligned so that it may be assigned to a pointer to any type of object with a fundamental alignment requirement and then used to access such an object or an array of such objects in the space allocated (until the space is explicitly deallocated). The lifetime of an allocated object extends from the allocation until the deallocation. Each such allocation shall yield a pointer to an object disjoint from any other object. The pointer returned points to the start (lowest byte address) of the allocated space. If the space cannot be allocated, a null pointer is returned. If the size of the space requested is zero, the behavior is implementation-defined: either a null pointer is returned, or the behavior is as if the size were some nonzero value, except that the returned pointer shall not be used to access an object.

Section 7.22.3.4:

The malloc function allocates space for an object whose size is specified by size and whose value is indeterminate.

The standard literally could not care less if there's a kernel underneath. It doesn't care how malloc is implemented or what the kernel, if any, looks like "from its perspective". The perspective that has relevance is that of the caller of malloc. From your own quotes:

> The pointer returned if the allocation succeeds is suitably aligned so that [...]

> If the space cannot be allocated, a null pointer is returned.

If you use mmap and "from your perspective" that's considered success then it's your perspective that's faulty. The standard is literally telling you right here^ there are 2 possibilities: either you allocate the space and return a pointer to the space, or you don't and you return null. There is no third option of "space cannot be allocated but you return non-null anyway". That's quite literally the end of the story.

The space IS allocated, it's just that this space (virtual memory) is not backed by physical RAM.

Try telling that to the C standard. It has no notion of virtual or physical memory. If it's allocated that means you can read/write to it, end of story.

> It has no notion of virtual or physical memory.

Exactly! That's why once virtual memory is allocated, malloc() is allowed to consider the operation successful. The standard does not care at all whether it is virtual memory allocation or physical memory allocation. It is completely unspecified in the standard what sort of memory must be allocated. So no spec in the standard is being violated by returning non-null pointer for virtual memory allocation.

You can't be serious about "every part". This definitely does not violate section 1.1 which only talks about the scope of the standard. It also definitely does not violate 1.2 which talks about what is not within the scope of the standard.

So once again, can you cite the exact section number from the standard that you think is being violated here?

See reply to sibling comment.

There is a deep irony in folks choosing to implement one of the few things the C standard DOES clearly define by papering over it with an alternative memory model.

Ironically, it's a memory model that greatly reduces the performance of modern systems, and als impacts its safety.

> [...] Linux blatantly violates the language specification [...]

We can't expect a kernel to be aware of all language specifications.

Yes, I know that in this case C is both the program's and the Kernel's language in this example but even then. Languages go through iterations (versions) and a Kernel can't be required to obey all languages which might run under it.

If you're suggesting Linux did not intend its malloc implement the corresponding function in the C standard then you're just lying to yourself. That's exactly why it's there and that's exactly what it's used for.

> If you're suggesting Linux did not intend its malloc implement

Linux (the kernel) doesn't have malloc. It's part of the C library, which is a completely separate project. What Linux implements is brk and mmap.

Playing loose with the word Linux here. Play along a bit maybe?

Linux is a kernel, not a C runtime so that's not really relevant. You say you realize that but then what are you arguing for exactly? Arguably you could complain that the glibc (or whatever libc you're using) is not working around that by, for instance, scrubbing the pages in malloc() to force the kernel to allocate memory.

But even then I'm not convinced that anything here is non-standard, at worse maybe we're in a bit of a grey area. As long as the kernel maintains its smoke and mirrors whether and how it allocates memory is irrelevant from the point of view of the standard. The C language has a rather simplistic memory model, it doesn't impose a lot on the implementation.

Now the problem occurs when the program attempts to access virtually-allocated memory and the kernel realizes that it can't find any physical memory to map it to. In this situation several things can happen but in general the process will be killed. Is it against the standard for the OS to kill a program for arbitrary reasons? I can't imagine why. It could also freeze the program, waiting for more memory to become available. Again, not against the standard as far as I can tell. Or maybe kill some other program to free memory.

If you have some specific part of the C standard in mind please do tell, I always find these language lawyering arguments interesting, somehow.

> If you have some specific part of the C standard in mind please do tell

See here: https://news.ycombinator.com/item?id=20145604

Also note the POSIX standard:

Upon successful completion with size not equal to 0, malloc() shall return a pointer to the allocated space. If size is 0, either a null pointer or a unique pointer that can be successfully passed to free() shall be returned. Otherwise, it shall return a null pointer and set errno to indicate the error.

In neither case is there any provision for returning a non-null pointer to anything other than an allocated block of memory of at least the given size.

Thank you for your reply but I still don't buy it. As far as the program is concerned it is returned a memory block, what this "memory" is effectively behind the scenes is none of the standard's business. As long as the implementation manages to maintain the illusion it's perfectly fine AFAIK. The problem is when this breaks down and the kernel realizes that it can no longer maintain the masquerade. If at this point it did something stupid like map this block nowhere or remap something that's already been allocated and let the program continue like this, then yes that would be a clear violation of the contract. But it does no such thing, it kills the program instead. If there's no program then there's no problem. At no point did a single C instruction get executed in an environment that did not maintain a coherent memory model. >In neither case is there any provision for returning a non-null pointer to anything other than an allocated block of memory of at least the given size. A pointer to virtual memory is a pointer to memory. The rest is an implementation detail. If, when dereferenced, the kernel issues an order to Amazon for more RAM and waits for it to be installed to resume the execution, that's none of the C standard's business.

> If, when dereferenced, the kernel issues an order to Amazon for more RAM and waits for it to be installed to resume the execution, that's none of the C standard's business.

We're not, and we never were, debating the situation where dereferencing the non-null pointer returned by malloc succeeds but takes long time due to your Amazon order. We've been talking about the situation where it fails. malloc is not allowed to return a non-null pointer to a memory block that cannot be written to. Linux does it anyway, and in doing so blatantly violates the standard.

That is my point, it doesn't fail. Either the kernel finds out a way to map the memory and it succeeds, or it kill the program and the instruction never runs. Code that doesn't run can't violate the standard. When the code is allowed to run all the invariants are guaranteed to be respected. If I write this code:

    char *b = malloc(2);
    if (b == NULL) {
      return 0;
    }
    
    b[0] = 'A';
    b[1] = '\0';
    
    printf("%s\n", b);
    
    free(b);

The standard tells me that if the malloc succeeds then the following code, if allowed to run, will display "A" on stdout. The C standard cannot and does not guarantee that a C program can't be interrupted however. For the sake of the argument we could imagine a kernel that instead of killing the program freezes it indefinitely on disk waiting for RAM to be available. It's functionally the same thing. As long as the kernel doesn't let code run with broken invariants it's fine. This is completely outside of the scope of a language standard to define.

Or, to try one last time from a different direction, if you consider that the C standard mandates that accessing memory returned successfully by malloc has to be successful and I happen to press ^C when that happens in a program, should the kernel refuse to kill the program? This is obviously absurd, but it's effectively the same thing: the kernel reacts to some external state and decides to terminate the program.

Okay this is a far more reasonable argument but I'm not convinced it's right. The standard does define normal and abnormal program termination. It also defines <signal.h>. SIGINT addresses the keyboard case (or the implementation can provide another signal). SIGABRT, SIGTERM, etc. are raised upon termination. For the keyboard case, then the program would be made aware, and it can indeed ignore the Ctrl+C request if it desires. That's perfectly normal and nothing absurd. For other types of termination, the other signals are raised. But in this case the program is terminated before any signal is raised at all. Now, as a practical matter I would argue the hosted environment should still be able to kill the program in the face of user request simply because nobody wants a host that's the slave of the program even if it's against the standard, but this is going far, far, far beyond that. It's happening at the request of neither the user nor the program; it's just happening because the host feels like it. Now that's both a violation of the standard and an uncool one at that!

P.S. I don't think indefinite hold is "functionally the same thing" as termination. A caller system(), for one, would need to return in one case, but not the other.

If a tiny, strictly conforming ISO C program obtains a tiny block of memory from malloc (like a few hundred bytes) and crashes when trying to initialize it, that is almost certainly a non-conforming ISO C implementation. The reason is that the implementation cannot support even one single program that exercises each of the implementation limits.

See 5.2.4.1 in http://www.open-std.org/jtc1/sc22/wg14/www/docs/n1548.pdf

However, it's true that not any old instance of this malloc problem demonstrates such a nonconformance. If it happens in a large program that has allocated gobs of memory, then no.

Basically if the system is low on memory that it can no longer support the execution of a small C program with modest memory use, then it becomes nonconforming.

However, the mere property that memory can be doled out by malloc which might later not be used doesn't make it ipso facto nonconforming.

Moreover, a system with any kind of memory management (including management that earnestly reports null for "out of memory") can be come a nonconforming C implementation if it is low on memory.

Allocated "space", i.e., virtual memory space. Malloc allocates virtual memory space, not physical RAM.

C has no notion of virtual memory. If memory is allocated for you that means you can read and write to it.

There is no guarantee offered by the standard that you must be able to read and write to memory allocated by malloc(). What if the memory was allocated and just a split second later, the physical RAM burnt out due to overheating? How is the standard supposed to guarantee reading and writing to it then?

Both the hardware burning after memory allocation and lack of availability of physical memory after memory allocation are outside the scope of the standard. The standard says nothing about them. A C program can fail in these scenarios without violating the standard.

> Linux is a kernel, not a C runtime so that's not really relevant.

Maybe it's more accurate to say that Linux is a kernel that assumes many features and semantics of the C runtime. But it certainly seems more deeply intertwined than you're willing to address here.

The C spec actually states at the beginning, "This International Standard does not specify ... the size or complexity of a program and its data that will exceed the capacity of any specific data-processing system or the capacity of a particular processor;"

Yes and that's why malloc can return null.

> Linux blatantly violates the language specification with no remorse.

We may be able to argue that it doesn't. ISO C says in 1. Scope, this:

This International Standard does not specify

[...]

— the size or complexity of a program and its data that will exceed the capacity of any specific data-processing system or the capacity of a particular processor.

It seems that these weasel words have an interpretation that can be bent around overcommitted memory allocation.

No it wouldn't. They clearly included the ability to return null to allow for when the data exceeds the capacity of the system. There is no provision for returning a pointer to memory that cannot be used.

You might think, but that's not how how conformance (of an ISO C implementation to the standard) works.

For an implementation to be deemed conforming, it just has to be demonstrated to successfully translate and execute one program that tests each of the minimum implementation limits.

See 5.2.4.1 in http://www.open-std.org/jtc1/sc22/wg14/www/docs/n1548.pdf

Only if no such program can be found can we then conclude that the implementation is nonconforming (and if the reason for not finding such a program is this overcommit issue, then we can blame that issue).

Your Linux system is indeed nonconforming if it is so low on memory that, for instance, no C program can allocate a 65535 byte object (that can be actually initialized, and used: a real object). Basically the memory situation has to be so severe that it takes the implementation below the minimum limits, whereby we can clearly demonstrate nonconformance.

Seems like a needless feature, overcommit, "oh let's make our app faster by pre allocating 2GB of RAM!" says the naive programmers, "Oh let's make our operating system more powerful by allowing overcommit.". Overall, you moved the allocation delay from the app to the OS, which has much less end to end knowledge of how to do it effectively, overcomplicated both layers, achieved practically nothing overall, like a dog chasing it's tail, except now both layers have more garbage code, so it's like an obese dog chasing it's tail!

Reply on news.ycombinator.com