MMU gang wars: the TLB drive-by shootdown

On some architectures, yes, but x86 has the dirty bit in the PTE for this; if it's cleared you're out easy (but that's unlikely); if it's already set I honestly don't know how costly it is to clear. You don't necessarily need to shoot the TLB on other CPUs though since the mapping is still valid; if there is some easier way to get the D flag cleared you can use that.

> But if the page is still in the TLB, then writes to it won't fault.

They can, a TLB entry has the full range of PTE flags to work with, e.g. a page may be flagged read-only for the kernel to do copy-on-write handling. I have no clue how exactly the dirty flag is set though :/ — I only know the function is indeed provided by the MMU itself, not a page fault.

You would effectively need to run different address spaces in different threads of the "same" process, which might interoperate badly with whatever guarantees the kernel provides or relies on elsewhere based on the assumption of having a unified address space for the whole process. Though I absolutely agree that this is worthwhile, it probably needs to be designed carefully and can't just transparently replace any and all uses of munmap().

I work on an allocator, and I've lobbied (Linux) kernel people for this over the years. I think everyone mostly agrees it'd be useful (and, the TLB shootdown cost is indeed often nontrivial), but the current architecture of the various subsystem's you'd need to touch makes it hard to do.

As allocators move more and more to a hugepages-first world, I think we'll see fewer and fewer benefits from faster unmappings (instead, we'll just spend more effort making sure we fill in the holes in existing in-use ranges).

That does exist, it's called MADV_DONTNEED and most operating systems have implemented it. However, MADV_DONTNEED on Linux was incorrectly implemented and (from memory) would always result in the page being unmapped immediately -- making it roughly equivalent to MADV_FREE. To quote the man page:

> All of the advice values listed here have analogs in the POSIX-specified posix_madvise(3) function, and the values have the same meanings, with the exception of MADV_DONTNEED.

You don't really need a separate API. The kernel can implement munmap() lazily, it just needs to also ensure that the address range isn't reused until a TLB shootdown is completed. LATR is a system that leverages this to implement lazy TLB shootdowns for munmap(): http://www.cs.yale.edu/homes/abhishek/kumar-asplos18.pdf

And in general, how contiguously you allocate memory plays a big part in whether freed memory can be easily discarded from the cache. If you get the heap to be exactly like a stack, then the cache shouldn't have problems. But I'll admit I'm not an expert and I could be missing something.

Almost always.

You can almost always answer the question "am I going to need memory again after this free()".

If memory is tied to a request or some other object you almost always know that.

You can almost always make a good guess at how much data a repeated operation will need, thus how much memory to reuse.

You almost always know the benefit of caching data in memory that is otherwise on disk. Often the OS is better at this than the developer thinks!

You almost always know if data is likely to be needed in memory again. When you free() you lose the space and the data. Quite often you can benefit from a managed free() that you can ask for the memory again still populated if nothing else needed the space it in the end.

You almost always know the security concequences of reusing memory. e.g. the OS cannot tell if calloc or malloc is needed. Secure(ish) allocators are a thing.

You almost always know if a memory block might grow or is fixed size.

You almost always know when you need an i/o buffer that can be reused.

You almost always know if smart pointers are beneficial or not.

In C its perfectly normal to allocate on the stack rather than the heap because you know better than malloc().

You often know more than the allocator about memory alignment.

I think you could almost always do better than the OS allocator when writing servers.

I'd agree its often not worth the effort.

Unless you are writing lots of C. free() is such a pain.

You can almost always write better, safer, faster memory allocators than ad hoc use of malloc() and free().

A side reason is that it'd probably heavily complicate the top level TLBs to have "please flush yourself" coming from anywhere other than the core they're attached to, and those are in the critical path between L1 and L2.

Totally spitballing here FWIW, I haven't been part of the design process for a TLB.

Yes, that's right.

Let us conceive of an inclusive cache architecture, just for the sake of argument. L2 already maintains a directory listing of all the lines which are present in L1I and L1D, and forwards cache coherency traffic to those caches based on the messages it receives. Expanding this to the ITLB and DTLB would be exactly the same circuits. The hard parts are already done.

But I think you're onto something with the hardware-managed dirty bit. Its much less general than the S->E->M transition, and doesn't need the Exclusive state at all.

I've done a bit more digging in the background. Turns out that ARMv8's TLB invalidate instructions are broadcasted operations. You don't need IPIs to execute them on every core. So processors do have some snoopy and/or directory management hardware interface for managing TLB shootdown. It just isn't as general-purpose as the rest of the cache maintenance hardware.

"Allocation" in this context just means open(), mmap(). A "shared memory pool" is a thing I would have no use for.

But I can assure you that you can have excellent, non-trivial parallelism with separate processes and chosen shared memory pages -- much moreso than with threads that must battle one another for access to locks, queues, and "pools". I routinely get order-of-magnitude performance improvement by reorganizing this way.

The single-writer ring buffer is the component that makes it all work. The environment might seem stark, but in exchange you can have exactly 0% concurrency overhead. Not wasting 90% on pool management and thread contention means other optimizations become meaningful. And, you can start and stop the processes independently.