Hacker News Reader

Comment by flohofwoe | Hacker News Reader

Memory allocators are already returning more or less "random" pointers, so using pointers for sorting is already a bad idea. Even using pointers for equality-comparison isn't usually a good idea because the same memory location might be reused when the allocator recycles memory (those tags could actually help with making such "recycled" pointers unique over time).

View on HN

Using pointers for sorting is fine as long as you don't care about the order (parent referenced std::set and it's not uncommon to use a set of pointers if you just want to store some objects). Likewise it's fine to use pointers for equality-comparisons as long as the pointer is valid (of course you have to dismiss the pointer when the memory is freed).

I'm pretty sure this memory extension doesn't affect uniqueness of pointers... that would indeed break a lot of software ;-)

> uniqueness of pointers...

What I mean is for example: you get a pointer by allocating memory, then free that memory, then you allocate again and get a pointer to the same memory location. Without a tag those two pointers would be equal, even though they come from separate memory allocations. This can be a source of bugs if the pointer is also used as some sort of object id, and I know that at least I stumbled over this embarrassing problem more than once in C++ until I switched to tagged handles :)

You can use the pointer as an object id, but only for living objects. The pointer becomes invalid the moment the memory is freed. Your bug was that you used an invalid pointer.

But doesn't this feature exactly help with this bug? Two memory allocations should now return different pointers (because the MTE nibble is different) therefore the comparision should fail. Unless ofcourse the application was using this undefined behavior (which would be very buggy since malloc can randomly refuse to reuse the same address even though it was freed and therefore create extremely hard and difficult to track bugs).

> Your bug was that you used an invalid pointer.

Exactly, but just declaring that a pointer is "invalid" doesn't help much with debugging such dangling pointer problems, while tagged pointers do.

It's not like an "invalid pointer" is any different from a "valid pointer" when you look at it. With the extra tag bits, an invalid pointer can actually be identified as such.

Be that as it may, existing software that used to work will no longer work, and the developers might not be around/able to fix it.

There are also cases of custom suballocators or arrays of objects - Looking at an address makes it possibly to figure out which array it belongs to. This code would break.

Granted, it would still be possible to do all this if you just mask off the tag bits, but it requires a software change.

The "suballocate out of some arrays" code should not break, because the whole array would be allocated at once and so would have the same tag for the whole range. Code that does a simple "is this pointer value inside the "array_base + size" range" continues to work, because array_base has whatever tag malloc() handed out for that array, and so do the pointer values that the suballocator handed out. I think for MTE to break your code you would have to be doing some pretty weird stuff with pointer arithmetic (beyond just the usual "technically maybe this is undefined behaviour but it works" level stuff).

It's always the case that some software that does things that are not valid-by-the-language-standard might break if run on a newer version of the OS or a newer system library version (remember the big flap about glibc memcpy() changing its behaviour when called for overlapping regions?). You don't want to break lots of software gratuitously, but sometimes the tradeoff is worth making.

Reply on news.ycombinator.com