The Arm64 memory tagging extension in Linux

The Mac IIci was the first 32 bit clean machine.

Plenty of dynamically typed languages have used the bottom bits in an address to store tags. You have two available on a 32 bit machine, three on 64 bit. The SPARCv7 has instructions that will do arithmetic on tagged values.

" ... if memory serves ... ", well done.

The extra bits stored flags that indicated whether handles were movable, could be purged at will, or came from a resource (on disk)

IIRC Excel went a bit further by using the leftover bits and even bit(s) starting at bit 24. That worked until hardware stopped wrapping around memory. Apple created a launch error “program had special memory requirements” which actually meant “this is Excel version so-and-so”.

And yes, that can bite you in the future, but the gains today can (appear to) outweigh that.

It also is perfectly valid when you control both the side that determines the constraints and the side that runs extremely close to them. For example, Cocoa’s tagged pointers (https://www.mikeash.com/pyblog/friday-qa-2012-07-27-lets-bui...) are fine, given that Apple both controlled the memory allocator and the library assuming it aligned allocations (they also are safer in that they won’t try to read from addresses that don’t point to valid memory. That’s essential for their use)

While conceptually similar (and made me think of the same thing), this doesn't really look like the same thing. In the Mac case, it was (ab)using technically unused addressed bits in unintended ways. In the ARM case, it is an explicit architectural feature. More like pointer tags in Lisp machines.

The biggest problem there was programs hiding away state and then not masking it out when using the addresses. This meant they couldn't just use the lower part of memory on new chips, they were completely incompatible. The lesson has been learned from that, and CPUs don't ignore reserved bits.

But also this is a hardware feature so the code will continue to be compatible anyway.

Even if this acts like a permanent truncation of the address space by 4 bits, so what? If you think you can keep Moore's law going strongly enough to hit that cap, you're probably going to hit the 64 bit wall soon enough after; might as well focus on that shift to 128 bit addresses

It does, as proven by Solaris on SPARC ADI, a similar technic with almost one decade of use by now.

IMHO it's less of a security feature, and more of a debugging aid (the overlap with security is when a failed check causes program termination). You might get more fine-grained overflow checks (vs guard pages), or dangling pointer checks (depending on how that key is stored, e.g. more than one key per memory page?).

It's faster then doing the same in code (e.g. via 'tagged handles': https://floooh.github.io/2018/06/17/handles-vs-pointers.html), or memory debugging tools like Valgrind or clang ASAN (and unlike Valgrind or ASAN, the checks are always enabled, also in the "release version").

In the most extremely sensitive environments:

1. It still helps with defence in depth

2. The threat of detection prevents attackers that need 100% stealth

3. Failures are likely to be detected which is still valuable as you can now mitigate damage

4. Failures are likely to be detected which helps get software faults fixed - which allows the ecology of security to improve over time

Is this really still a problem with virtual memory management? The virtual address space size could be a property of the process (e.g. 48 bits vs the full 64 bits). The vast majority of applications today doesn't even need more than 32-bits of address space and could use the top 32-bits for something else if the memory mapping hardware ignores the unused bits).

2^48 == 256TB.

Anyone needs more than 256 Terabytes of RAM?

The estimated number of atoms on earth is on the order of 1e50, which is 60 bits. If you take the 4 bits out of the 64bit address for tagging, you can still address individual atoms.

EDIT: My math is bad, see comments below.

I can unsarcastically say that no one will ever need full 64bit addresses on this planet.

Memory allocators are already returning more or less "random" pointers, so using pointers for sorting is already a bad idea. Even using pointers for equality-comparison isn't usually a good idea because the same memory location might be reused when the allocator recycles memory (those tags could actually help with making such "recycled" pointers unique over time).

Comparing pointers not coming from the same allocation is undefined behaviour per C++ standard.

I'd assume the key is stored in the hardware memory translation structures like the TLB.

mwsealey's comment on the LWN article (in response to somebody asking that question there) has a good explanation of it. Basically exactly where the keys are stored is up to the implementation, but one reasonable implementation is for the memory controller to put them in the top part of the system DRAM (which is then not made available as 'normal' RAM). Architecturally the allocation tag space is entirely separate from system memory, though, and the only way to get at it is with the provided instructions for reading/writing tags, so any particular implementation can put any kind of storage it likes underneath. (A software model, for instance, might just allocate a separate lump of host RAM for storing the tags. QEMU's MTE emulation does that.)

It seems that the tag mappings will need to live in separate physical RAM, so that info could be pulled from the tagmap RAM chips at the same time as the data is pulled from the data RAM chips.

Using multiple RAM chips to increase throughput is an old computer engineering technique; it was, for instance, the entire justification for planar graphics modes in early graphics hardware.

You're overlooking a) existing page table lookup overhead b) hierarchical caching.

Correct. x86-64 mandates “canonical” addresses where all the upper bits must be the same in order to dereference a pointer.[0] So on a processor with 48 physical address lines, the upper 16 must be either all set or all unset. This is obviously to prevent people using those upper bits to store extra data.

Of course, this doesn’t stop you from being able to[a], it just makes it harder.

[a]: The processor doesn’t know the “type” of a register’s value; It’s just bits. It could be a pointer, integer, floating point, etc. until you tell the processor to do something with it. So one could obviously use those extra bits to store data, then when a dereference is needed, store those bits in another register, dereference, then put the bits back. But that’s a lot of work to save a byte or two.

[0]: Intel SDM Volume 1 §3.3.7.1:

> In 64-bit mode, an address is considered to be in canonical form if address bits 63 through to the most-significant implemented bit by the microarchitecture are set to either all ones or all zeros.

Absolutely, or in the low bits of word aligned pointers as someone else mentioned above.

Emacs Lisp is an example; here's its tagging apparatus.

https://git.savannah.gnu.org/cgit/emacs.git/tree/src/lisp.h#...

And common javascript engines. Turns out a lot of interpreters hide bits away in what they precieve to be unused address bits.

Its one of those lessons that everyone seems to be constantly relearning. Address spaces grow.

One of my first "real" jobs, when this topic came up a very senior person said something to the effect. We continue to find ways to use up one of those bits roughly every year. Moving from 32-64 bit VA's is going to get beyond the end of my career but someone will have to deal with it in the future. So its a little slower than 1 bit a year, but everyone is adding a few more bits (52/56/etc) because there are actual commercial computer systems with a few tens of TB of ram today. If persistent memory ever becomes a thing, someone _WILL_ want to mmap their multi PB storage cluster, and then there will be a scramble for more address bits.

No, like on most other archs, people use tagged pointers in lisp and other runtimes on amd64 all the time. But the machine (like most ISAs) doesn't ignore any bits so you have to clear the tag bits before dereferencing.

You may be thinking of the fact that a full 64 bit VA space isn't supported in current implementations. But this has no impact on use of tagging compared to most other archs.

The keys are stored in the bottom nibble of the top byte of pointers (I.e. bits 56-59)

It’s similar, but CHERI provides much stronger guarantees. (The ARM implementation will use 128-bit pointers, with the top half being used to store much more granular keys.)