Hacker News Reader

Comment by cmbaus | Hacker News Reader

My memory is a bit hazy in this area, but I think by default memory is over committed in Linux. What that means is malloc() can return an address that doesn't have physical memory assigned in the page table. Memory isn't committed until it is written to.

This isn't the case with the default MSVC implementation of malloc() in Windows. In Windows address space is reserved and committed with VirtualAlloc(), and typically that is done in one step.

I think memory is over committed because Linus wanted to keep the memory footprint lower than NT early on in the development of the kernel. The drawback is applications may segfault when writing to memory that was successfully returned by malloc().

View on HN

You seem to have some terminology problems here. Windows VirtualAlloc may "commit" memory but that does not mean it actually reserves physical pages [1]. That always happens only when the memory is accessed. On the other hand, MSVC's malloc() probably uses HeapAlloc(), which in turn uses VirtualAlloc(). I don't think there are any fundamental differences between Linux and Windows here.

[1] "Actual physical pages are not allocated unless/until the virtual addresses are actually accessed." http://msdn.microsoft.com/en-us/library/windows/desktop/aa36...

That link says: "Allocates memory charges (from the overall size of memory and the paging files on disk) for the specified reserved memory pages."

It does count against the total memory allowed. My laptop has 8GB of RAM and 1GB page, giving me a 9GB overall commit limit. If I spawn a process that eats up 1GB at a time, even Task Manager can clearly show me going up and hitting 8/9GB, then I'll get OOM in my process.

Windows won't commit memory that a process can't use. You can't overcommit, although you might end up in the pagefile. Without the odd concept of fork, you don't end up with processes having huge "committed" address spaces that aren't ever going to be used.

The note about physical pages is just saying it's not mapping it, not that it's not guaranteeing it.

Right. I stand corrected.

VirtualAlloc can reserve address space and commit pages for it depending on the flag provided. Address space is a resource of the process, and pages are a resource for the entire system.

Reply on news.ycombinator.com