Ideally every application that is unlikely to use more than 2GB of memory should use the X32 ABI.
Cache latency is the most important bottleneck of today's general purpose CPUs. X32 Helps a lot with that and not just in "synthetic" benchmarks.
So all the hassle of maintaining a separate ABI (toolchains, kernel, libraries etc.), for a modest performance increase for some subset of applications?
Starts to smell pretty niche to me.
In principle yes, but it is hard to predict which code will be at some point performance critical and which code will not. In the worst case you chain a lot of alleged "non performance critical" tasks together and in the sum (actually product, performance percentage is multiplicative) your problem becomes a performance problem.
> Or might want to mmap() large files
This is a fringe application that is not covered by my definition of "application that is unlikely to use more than 2GB of memory". Most programs read linear from files.
> So all the hassle of maintaining a separate ABI (toolchains, kernel, libraries etc.), for a modest performance increase for some subset of applications? Starts to smell pretty niche to me.
As far as I see it there is not enough awareness of this feature. So instead of declaring it "niche" that causes "hassles" maybe advertise it more since the hard work (toolchains, kernel, libraries etc.) has already been done. (My personal use case is derivatives market simulations and performance the gain is in the double digits percentage. I'm pretty sure other use cases exists, they just need to found.)
In the end, who defines "hassle", how big is it really, and who defines "niche"? Potentially accelerating 90% + x of code running on your CPU is hardly a niche.
Hardware is typically a price sensitive netbook contrained by RAM and with a low powered x86-64 CPU, so that 13% performance increase might be noticeable.
The software is a design that is inherently multiprocess for security reasons (where every webapp is essentially a browser tab running in its own sandbox).
My guess would be no. For a sort-of counter-example, Apple started migrating to a 64-bit environment for their smartphone and tablet systems already back in 2013, and I think 64-bit Android systems are also pretty common nowadays.
And, if they wanted X32 support, I think it would be more than a "couple of commits in their issue tracker"; AFAIU V8 has no X32 support, which would definitely be a non-trivial undertaking. Do you think they would commit to implementing and maintaining V8 X32 support, just to get a modest performance improvement in such a niche market? Also not to mention that for security reasons they might prefer 64-bit anyways (see ASLR).
My opinion is that 32-bit is rapidly dying for anything approaching "general purpose" (including things like smartphones etc.). For embedded system, sure, 32-bit systems are under no threat of extinction; heck neither are 8 or 16-bit microcontrollers under any particular threat. For an example, look at RISC-V; they haven't even bothered to bring up Linux support for the 32-bit variant, RV64GC is the baseline they are targeting. 32-bit RISC-V seems to be targeting microcontrollers or maybe some simple embedded OS's, but not Linux.
Huh? So if I chain together tasks A->B->C, and assuming (optimistically) that X32 completes a task in 90% of the time of X86-64, then the total time of X32 is not .9x(time_A+time_B+time_C) but, what? .9^3(time_A time_B time_C)? I'm lost, can you explain?
> As far as I see it there is not enough awareness of this feature.
Hmm, I'm not sure. Seems it has been discussed extensively all over the place, certainly enough that people interested and able in shaving some (likely) single-digit percentage time of their application, that they have determined is due to mem/cache usage of pointers, ought to be aware.
> the hard work (toolchains, kernel, libraries etc.) has already been done.
That is true, but the point is that there is an ongoing maintenance burden. If almost nobody is using it, is it worth it to keep paying the maintenance cost? Evidently the Linux kernel maintainers are starting to think that nope, not worth it.
> In the end, who defines "hassle", how big is it really
Presumably the people who have to maintain it?
> and who defines "niche"
Number of users, maybe? Did you read the lkml thread? The X32 Debian maintainer popped in, and mentioned that per popcon X32 in Debian has 7 users. Not 7k, 7. Seven. Now there are certainly users who are not enabling popcon, but still. For comparison, for x86-64 popcon reports 172k users. So for every user who thinks X32 is a good idea and what they need, we have 24k users who prefer x86-64 (certainly a large fraction of those implicitly, but still).
Lets assume Instruction I_n is dependent on I_n-1 is dependent on ... I_1 (I_1->I_2->...->I_n). The Instuction I_i takes t_i amount of time. Lets further assume those Instructions can be pipelined and parallelized hence the number of executions of the full Instruction series is big compared to n. Then the time needed to execute the series is max(t_i). If we have a optimization that modifies all the times to o_i * t_i the new time needed to execute the series is max(o_j * t_j) hence the time ratio saved on the whole series is r = max(o_j * t_j)/max(t_i) and the time saved per execution is then n * r because the time you save to execute the whole series is not dependent on n itself.
> Debian maintainer popped in, and mentioned that per popcon X32 in Debian has 7 users.
In order to do it right you have to distribute X32 versions for packages where it makes sense in your otherwise X86_64 distribution. Then you have increased the performance for everybody and an adoption of 100%.
It is absolutely idiotic to have 64-bit pointers when I compile a program that uses less than 4 gigabytes of RAM. When such pointer values appear inside a struct, they not only waste half the memory, they effectively throw away half of the cache. -Knuth
This is a discussion about whether the kernel itself wants to run using x32 mode, which is arguably much less useful. Such a system would be limited to 4G of physical memory, etc... There aren't that many such systems left in the x86 world (that run Linux, anyway -- I'm literally working on an x86_64 port for Zephyr now, and it uses x32).
Such kernel will run x86 binaries (in compat mode), x86_64 binaries and x86_32 ones.
If the kernel removes X32 support, I guess it won't take long for toolchains to remove it either. Why should they pay the maintenance burden of it if nobody uses it?
> This is a discussion about whether the kernel itself wants to run using x32 mode,
No, the kernel has never supported running in X32 mode itself. This is about the compatibility syscalls allowing a X32 userspace with a x86-64 kernel.
It's often useful to be able to address in 64bits even if you are never going to use 2GB of memory. And it's probably more and more true the more you care about performance (e.g., mmap() and friends). So I don't think the set of useful workloads that get a performance boost from smaller pointers and don't get hindered by a small address space is as large as you are assuming.
https://link.springer.com/chapter/10.1007%2F11688839_14?LI=t...
found speedups of 13.4% for integer workloads. That's significant for my space, hence my interest.
Using a single 1G page per process means switching is faster, and might have other benefits in a post-spectre world...
I was replying to saying that most apps should use x32 if they don't use much memory. That sounds like a nightmare in a general purpose distribution, with having every library duplicated and interoperability issues.
I get the part about duplicated libraries, but I'm not certain it's that big of a deal: Most people don't have more than 2G of code so it should be possible for the 64-bit libraries to be used (and just mapped low) with an m32-built application, if the things that create new pointers can be limited to 32-bits.
One for x86_64, one for x32. And if you want to support traditional 32 bit application you'll need the i686 variant as well. None of these variants are ABI compatible with eachother. If nothing else, that's a burden on logistics.
The most common use case is to just mmap() a bunch of cache files and just fully outsource the cache hierarchy of those to the Linux VM subsystem which will do a much better job than almost any app even can (because it does global optimization across all processes).
> I get the part about duplicated libraries, but I'm not certain it's that big of a deal: Most people don't have more than 2G of code so it should be possible for the 64-bit libraries to be used (and just mapped low) with an m32-built application, if the things that create new pointers can be limited to 32-bits.
This is way too much complexity. "Things that create new pointers" are any functions that return new memory and there are too many of those. A general purpose distro will never take on that much complexity for a 13% speedup. If they wanted even a fraction of that complexity they'd compile x86_64 at several levels of supported instructions for bigger gains from just tweaking a few compiler flags.
I don't understand what you're saying here.
Why do you think having the pointers be 64-bit instead of 32-bit help if the program isn't using more than 2G?
Can you give me a specific example?
> This is way too much complexity. "Things that create new pointers" are any functions that return new memory and there are too many of those.
mmap() and brk() are the two big ones, and almost everything else derives from them.
The tricky part is to get MAP_32BIT into every call to mmap(). That means hacking up the dynamic linker, or having a personality flag that just pins it kernel-side. I don't think either is onerous though (probably a few hours work).
The program isn't using more than 2G of RAM it just mmap()s more than 2G of files and so needs enough address space to fit those mappings. I've come across that when doing caching for an image processing program. The simple way, that works very well, is to just have the thumbnails on disk and mmap() them in. With even medium sized collections it's easy to lazy load more than 2GB of files but that's fine because 64bit address space is cheap and the Linux VM will take care of throwing away things that haven't been used. Makes for a very simple memory+disk cache hierarchy that the OS manages for you entirely.
> mmap() and brk() are the two big ones, and almost everything else derives from them.
Sure but that's at the base of the call chain. Each library will then bubble up those 64bit pointers to it's callers. You can't have only 32bit versions because at least some part of the apps will need more memory so you're stuck having two versions of everything.
If you need the address space, then use 64-bit.
I don't need the address space.
If that was confusing, my apologies.
> Sure but that's at the base of the call chain. Each library will then bubble up those 64bit pointers to it's callers.
Which will have clear upper 32-bits, which means using %eax instead of %rax in main() will work fine.
Well obviously, but you asked for the use case. I was replying to someone saying x32 should be used everywhere 2G of RAM is enough. And I was pointing out there were other reasons 64bit was required and those can even be embedded in libraries in general usage. So I think x32 makes sense if you have a specific use case where the speedup is relevant but not for general usage.
> Which will have clear upper 32-bits, which means using %eax instead of %rax in main() will work fine.
I don't see how this will work without recompiling every library, which is what x32 does right now I think. Everywhere in the code there will be dereferencing of 64bit pointers and every pointer stored in every struct will be 64bits. So does it really matter if the upper 32bits are empty from a performance standpoint?
Yes!
main() would push %eax instead of %rax, effectively doubling your cache size (in words).
> I don't see how this will work without recompiling every library
The x86_64 abi uses registers for the arguments, so fopen() would receive file in %rdi, and mode in %rsi, but the caller only used %edi and %esi. When fopen() calls malloc() our libc knows we want 32-bit pointers, so it uses mmap(MAP_32BIT) so the pointer has the top bits clear and thus when fopen() returns the pointer in %rax, main() can still store it in a 32-bit pointer reading %eax.
Structs are certainly a problem. If main() tries to look e.g. at struct utsname, it would need to know (in this case) that these are "long" pointers rather than short ones. This pain would need to be handled by main() (for example, using wrappers of some kind), but it wouldn't require building or maintaining two sets of the libraries.
All that complexity is certainly not worth it versus just shipping a second set of x32 libs. At least that doesn't require those tricks and wrappers and is at least easy to maintain within the current multi-arch infrastructure distros already created to support 32bit binaries in 64bit distros.
> you can mmap parts of a big file no problem with large file support in libc.
For the speedups discussed here I definitely don't want to spend any time writing special code just to be able to read a 3GB file. That ship has long sailed and good riddance to all those subtle issues where things seem to work fine until they don't and it's not clear why.
Don't forget to not put the bugs in, though!
And as the kernel is responsible for caching (or what kind of caching do you mean?) You will not lose that.
So, if some programs in the system use the 64-bit ABI and some use x32, then memory usage may substantially increase. And, of course, this also affects CPU caches.
Seriously, most people I know just don't think that critically about 64-bit, and don't understand the performance gain of pairing 64-bit instructions with 32-bit pointers.