Hacker News Reader

Comment by delta_p_delta_x | Hacker News Reader

MTE is essentially hardware-accelerated AddressSanitiser[1].

[1]: https://clang.llvm.org/docs/AddressSanitizer.html

No MTE and AddressSanitizer are implemented completely differently under the hood and catch different kinds of memory bugs.

MTE tracks provenance of pointers which means it catches bugs where a valid pointer derived from one allocation is used to access another allocation. Provenance is indicated by a fixed number of tags available. So there’s a 7% chance of not detecting an occurrence of a memory bug.

ASan is implemented differently: it adds red zones next to allocations. In theory it could have a false negative if a pointer jumps over the poisoned region. But it works well for stack memory in addition to heap memory. MTE doesn’t protect your stack allocated objects.

This is all true but it also is true that MTE was in part built to accelerate address sanitizer.

Kostya/et al who pushed for and designed the extension, was trying to accelerate address sanitizer so it could be on all the time. Among other things.

In fact, most presentations presented it literally as a way to do hardware accelerated ASAN (again, among other things), so the post you responded to is correct in that sense.

(I was there at the time, helping him figure out how to push for it)

Heh, you can kinda think of MTE as ASan except instead of a small range of a guaranteed redzone around heap pointers, it's a massive `2^56 * (random number, ≥0, on average 15)`-byte "redzone" (and some padding up to a multiple of 16 bytes which can predictably hide a bug, though at least such a bug won't corrupt unrelated heap). Stack handling is a significant difference though.

To expand a bit: 16-byte chunks of memory can be associated with a four bit tag. Then you steal four unused high bits from your pointers to store a tag value. When memory has a tag, a pointer used to access it must have the matching tag in its high bits. Your malloc implementation can then assign a different tag to adjacent allocations and any overflow into an adjacent allocation will have a mismatched tag and will trap. Likewise, change the tag on free and an attempt to use the pointer after the allocation has been freed will trap.

Of course, this doesn't come for free. Four bits per 16 bytes means a 3% increase in memory needed for tagged memory, hardware overhead for checking tags on memory accesses, and software overhead of setting/changing/clearing tags as necessary. This overhead is low (Apple shipped this in flagship hardware a year ago and nobody's complaining about performance there) but not zero, and an implementation with poor performance could be a real problem.

Implementing MTE is mostly another step in Apple’s efforts towards memory safety. There are bunch of things they’ve done to try to make this work a little better, but for obvious reasons, this is never going to be turned on by default for user apps and is only enabled for specific pathways.

> Because EMTE tag checking imposes a performance cost, we designed Memory Integrity Enforcement to take advantage of our secure allocators first and use EMTE to protect only smaller individual allocations within a type bucket, which software allocators can’t defend on their own.

Which seems to imply that outside of Apple’s targeted use in the kernel, the developer use of MTE on Apple platforms requires that you migrate your code to use their typed allocators…which I don’t know how many people are going to do that (and it is clear their answer is going to be to point people to Swift since it makes these types explicit for the compiler and even then, I am sure there are bugs in Swift in certain cases). Memory tagging is neat tech, but I don’t know if it’s going to gain all that much traction. It requires a burden that I suspect most app developers are not willing to take on and given that the user interaction is a crashy app. This is a path to nowhere except those of us who understand the trade-off.

Ive had MTE enabled on GrapheneOS for the past 3 years for pretty much all apps I use without any issues.

Theres a couple of apps that use C++ libraries for which I disable MTE.

Most android apps using Java or Kotalin which hugely helps with avoiding buggy apps crashing with MTE enabled.

Android can and does regularly force developers to make changes to their apps to improve privacy and security on the platform. Like theyve done with other changes they could gradually force developers to fix bugs in their apps. Maybe offer a MTE opt out for apps that use memory unsafe languages.

Think its likely Apple and Google will increasingly push forward the use of MTE in Android and iOS

Typed allocator migration just requires a recompile, it’s not a big job.

What does the malloc or hardware do if for some reason your program needs to access memory addresses that overlap your tag bytes ?

I know that it's a really improbable scenario and the OS would also just refuse you allocations at some point, but what would the malloc implementation and the MTE do in such a case ? Fail the allocation ? trap when reading the pointer since it would point to the "wrong place" ?

If you mean the memory that stores the actual tags associated with each 16-byte chunk of memory, that's a separate region that should be inaccessible except for using the special instructions for reading/writing tags.

4 bits per 16 bytes is ~ 1/4 the cost of ECC, which is possible to implement with just an extra cycle or two of latency in the memory controller. MTE seems similarly lightweight. Costs some transistors and a percent of a percent of power budget, but much like ECC it seems a fair bargain.

In case anyone wants to learn more how ASan works, I wrote a blog post on it at https://blog.trailofbits.com/2024/05/16/understanding-addres...

It also explains that the fact ASan works well with, e.g., std::vector in C++ doesn't come for free and if you want ASan to detect bugs when your custom collections or allocators are used you have to use special ASan API to mark [in]accessible memory.

Reply on news.ycombinator.com