Hacker News Reader

Comment by dbattaglia | Hacker News Reader

"Virtual addresses are the size of a CPU register. On 32 bit systems each process has 4 gigabytes of virtual address space all to itself, which is often more memory than the system actually has."

I guess this is not the most up-to-date document?

View on HN

>I guess this is not the most up-to-date document?

it's also not correct. It doesn't have all 4GB "all to itself", because a portion of that (usually 1 or 2 GB) is mapped to the kernel.

"each process has 4 gigabytes of virtual address space all to itself"

A process does indeed have all 4GB of VIRTUAL adress space to itself. unless I'm misunderstanding you.

Yes, VIRTUAL memory. Most operating systems (Windows, Linux) leave 2 or 3 GB for the user process and reserve the of the address space for themselves.

That way userspace-to-kernel switch does not require changing active page table (and also avoids switcharoo each time kernel needs to access userspace memory).

If you are thinking about the “which is often more memory than the system actually has" part, I don't know if it's outdated even today: the vast majority of Linux systems these days are Android phones, and I wouldn't be surprised at all if a good proportion of those didn't have more than 4GB of RAM.

I think that's probably still true for what 32-bit systems are still out there today.

And regardless, I think the majority of systems running Linux today are phones, which usually have 4GB or less of RAM.

But I expect the FAQ was probably originally thinking about desktop or server systems, so, yeah, the intent there is probably out of date. Those types of systems are rarely 32-bit these days, and usually have a bit more than 4GB of RAM.

> I think the majority of systems running Linux today are phones, which usually have 4GB or less of RAM.

Even this is quickly becoming less and less true (for new phones). Even the Pinephone comes with 3 GB of RAM at a $200 price point, and that's inflated because of the niche, low volume nature of its production.

Samsung's "mid range" A series smartphones, for instance, start at 3GB at the absolute lowest end, with most models coming with 6 GB of memory. I expect this will be even more common in a year or two.

My OnePlus 3T is nearly 4 years old now and has 6GB (and is really showing its age...)

What's the use case of having so much ram on a smartphone ? Gaming?

App-switching (multitasking) without LRU apps getting force-closed to make room for active apps. In other words, if you like to keep apps open, more RAM will reduce the chances of an app opened a while ago having to "start fresh" when you switch back to it, losing whatever state it had when you last used it.

Allowing app developers to not worry about optimization, put more trackers and more annoying ads...

I think there might be some more hardware-specific nuance here. e.g. /proc/cpuinfo says this on a couple of different x86_64 systems that I checked.

  address sizes : 36 bits physical, 48 bits virtual
  address sizes : 40 bits physical, 48 bits virtual

PS: I don't understand what this means, btw.

Your CPU can handle 39-bit physical memory addresses (up to 512 GB of physical memory), and 48-bit virtual addresses (256 TB). Your operating system maintains a mapping from virtual to physical addresses, usually arranging the map so that every process has a separate memory space. Pointers are all still 64 bits long though.

In practice the actual available usable address space for userland is 64 TiB due to user/kernel split and the kernel maintaining a virtual mapping of the entire physical address space (minus I/O ranges) [0].

However newer incoming 5-level page intel chips [1] will allow up to 57 bits of address space, 128 PiB in theory though in practice 32 PiB of userland memory. See also [0] for discussion on practical limit for 5-page too!

[0]:https://github.com/lorenzo-stoakes/linux-mm-notes/blob/maste...

[1]:https://en.wikipedia.org/wiki/Intel_5-level_paging

True, though /proc/cpuinfo only reports the size, which is ultimately what the CPU cares about. Plus the most relevant limit is what your motherboard and wallet supports, which is often far lower.

Indeed, and as you say, sensibly speaking you are hardly likely to hit those limits in any likely (esp. home) setup. The actual meaningful limit is usually the CPU physical one as home CPUs very often have stringent memory limits (often 32 GiB or so) and of course you rely on the motherboard's limitations also.

Having said that I did write a patch to ensure that the system would boot correctly with 256 TiB of RAM [0] so perhaps I am not always a realist... or dream of the day I can own that system ;)

[0]:https://git.kernel.org/pub/scm/linux/kernel/git/next/linux-n...

You're not the only one dreaming; I had to use >200GB of swap on my home system last year.

So are the 16 leftmost bits of a virtual address always 0?

They have to be same as the maximum addressable bit, i.e. in the case of 48 bit virtual address size the 48th bit.

This is actually kind of a cute way of dividing kernel and userland space as you just set the upper bit to 1 for kernel addresses and 0 for userland.

EDIT: Specifically talking about x86-64 here.

https://github.com/lorenzo-stoakes/linux-mm-notes/blob/maste...

No, it must be sign-extended from the top bit of the valid set. Otherwise the address is non-canonical.

This is true for x86-64, not true for other architectures such as arm64.

Apple uses the high bits to cryptographicly sign the pointer value.

Fascinating. Does this confer some of the benefits of ECC RAM, for pointer data only — without the hardware cost?

Some (but I believe the advantage is that it’s much harder to inject valid code from a buffer overflow).

PAC is more about CFI than preventing shellcode injection (which is done through codesigning and memory protection, mostly).

It's for a different purpose. (as in mitigate to some extent security bugs) And isn't an Apple feature only but an Arm one. (that is only rolling out on Cortex with Cortex-A78C and A78AE)

A paper on it from Qualcomm: https://www.qualcomm.com/media/documents/files/whitepaper-po...

And there's also MTE which is upcoming.

Hmm, it appears that the top byte on arm64 is only ignored if TBI (Top Byte Ignore) is enabled.

I don't think pointer signing requires TBI though. Pointer signing uses the PAC instruction to sign a pointer, and the AUT instruction to verify and unpack the signed pointer, but in its signed/packed form it is not a usable pointer. So actual addressable pointers need not support non-canonical addresses.

Apple runs PAC without TBI enabled, I believe.

Oddly enough the unused bits are in the middle of the address. They're also sign-extended rather than filled with zeros, so sometimes they are ones and other times they are zeros.

In the middle of the address _space_, I should say.

If you interpret your addresses as signed values, then the entire address space can be contiguous.

I'm not suggesting that this is a good idea, but is certainly an idea.

Negative addresses? Anathema!

Yes generally for userspace addresses they are 0. But more importantly they can be used for other stuff, commonly referred to as pointer tagging / smuggling etc.

It's a useful optimisation technique where you can add some extra metadata without having to dereference a pointer.

The reason why amd64 checks whether the addresses are “canonical” is discourage exactly this trick. On almost all platforms that simply ignored upper byte of pointer (m68k, s390, IIRC even early ARMs) this lead to significant compatibility issues.

As for storing tags in pointers on 64b platforms it is probably better to use the 3 low order bits. Another useful trick is what was used in PDP-10 MacLisp and is used by BDW GC: encode the type information in virtual memory layout itself.

I guess it checks it when you actually try to dereference the pointer? On Intel too you still have to "repair"the pointer before you use it. It's definitely not the safest optimisation but it can be used to great effect when needed.

I think Intel is adding CPU support for pointer tagging operations in the future which should make them a lot easier / safer / more efficient to work with, though I can't find a reference now, it doesn't refer to it as pointer tagging.

Any more information on encoding the type information in virtual memory layout? Sounds cool.

I guess you have different types allocated in specific regions?

Most general purpose ISAs (eg. SPARC and IIRC RiscV has something similar) with some kind of intrinsic support for tagged pointers also prefer the tags in low order bits.

And you are right that the tag inside address trick involves allocating objects of same type in different continuous regions. Usually such that whole page contains object of same type (as far as the tagging scheme is concerned) and by either masking off lower ten-ish bits of pointer you get to type header or you have some global out-of-line map of page frame->type.

It means that it can address 40-bits of address space worth of physical memory, but that virtual memory addresses can use 48 bits. Physical addresses are just your RAM bytes numbered 1 through whatever. Virtual address space is the address space of a process, which includes mapped physical memory, unmapped pages, guard pages, and other virtual memory tricks.

> Physical addresses are just your RAM bytes numbered 1 through whatever.

Not really. There are lots of holes in the physical address map. Look at /proc/iomem. Look at all of the gunk in there at addresses lower than the amount of RAM you have. Look at the highest “System RAM” address. It will be higher than the amount of actual physical RAM that you have.

On 32-bit systems, 4 GiB is indeed often more memory than the system has (think 512 MiB for some Raspberry Pis). And on 64-bit x86 systems each process has 256 PiB, which is also more memory than the system has.

Reply on news.ycombinator.com