Comment by _dcwr | Hacker News Reader

Comment by _dcwr | Hacker News Reader

I like the article and the tricks. I recall seeing a presentation by a game developer who mentioned something similar, something about intermixing actual data pages with something else (read only ones or even unusable ones?) to trigger an error in case of an out of bounds read/write that was far enough (which apparently happened often enough for this to be helpful).

The address space is not actually 64 bit and it's not just the OS limiting it. The CPU/MMU/IDK (I'm not low level enough to get all this stuff) does too in some ways. I assume these values mean that I can have 2^39 bytes of actual RAM and 2^48 bytes of virtual address space:

>[frex@localhost ~]$ cat /proc/cpuinfo | grep address

>address sizes : 39 bits physical, 48 bits virtual

>address sizes : 39 bits physical, 48 bits virtual

>address sizes : 39 bits physical, 48 bits virtual

>address sizes : 39 bits physical, 48 bits virtual

This also came up back when I was looking through PUC Lua 5.1 code for a paper and (out of curiosity) looking into code of PUC 5.2 and 5.3 to see the changes and into code of LuaJIT for comparison.

LuaJIT uses top bits of a pointer to store type info in case of light userdata pointers: https://www.freelists.org/post/luajit/Few-questions-about-Lu...

This reminds me I should probably do my write up of how PUC Lua 5.1 VM works and implements it's features in English finally. If someone's interested in any way I'd appreciate dropping me a word to let me know that there's market for that so I don't fear that I will be writing down stuff I already grok for no one to read and benefit from.

maybe this https://www.youtube.com/watch?v=qYN6eduU06s&feature=youtu.be... presentation

Yes, this presentation. I mistakenly thought it had to be a GDC one because it was about games.

> I assume these values mean that I can have 2^39 bytes of actual RAM and 2^48 bytes of virtual address space:

Yes. There’s no point having 64 physical address lines if 26 of them aren’t going to be used by anything

Where does the number 21 come from?

My brain said 64 - 39 is 21... my bad

One use is tagged pointers, using the top unused address bits for e.g. type information (https://en.wikipedia.org/wiki/Tagged_pointer)

LuaJIT is where I originally ran into this. It uses both NaN tagging with C doubles and tagged pointers with 64 bit pointers.

We're using this technique to stuff the file descriptor with the accompanying object for the epoll registration, so one can check if the (reusable) object was reused for something else while registered with other file descriptor in the epoll: https://github.com/sociomantic-tsunami/ocean/blob/ee4f78d527...

Amiga programming manuals were very clear about not (ab)using the top 8 bits even if the A1000/500/2000 were only equipped with 24 physical pins for memory (limited to 16M), so of course Microsoft Basic for amiga used the upper bits, and when the 68020+ cards arrived, it broke since the upper 8 bits actually pointed somewhere.

What is inconceivable today, will be reality at some point.

The same happened for Mac OS.

On amd64 (and IIRC also Alpha) hardware checks that unused bits of address are all equal to highest used bit (or IIRC all zero on Alpha) to discourage this kind of tricks.

And I vaguely remember that ESA/390 is actually 32bit address architecture in hardware and it's 31bitness comes from using highest bit of address as a flag that such tricks are taking place.

Reply on news.ycombinator.com