One of the less-well-known features of Linux is that you can do this! Theres's a thing called the "x32 ABI" (use the option -mx32 with gcc or clang; you'll need all your libraries compiled with it too) where:
* As far as the processor itself is concerned, the code runs in 64-bit mode, so you get the extra (and wider) registers from that.
* But pointers are still 32 bits, so you get the memory savings of 32-bit mode.
In principle, as long as you're using <4GiB of memory, it should be at least as fast as the best of 32-bit or 64-bit mode for any particular program. But I haven't heard of it being used much.