Every modern general purpose computer already has multiple layers of memory. This would just be an additional layer. The virtual memory subsystems in the OS will handle this. At the end of the day it's just caches all the way down. A workstation with 16GiB of "on-chip" memory would be like a huge L4 cache for the say 512GiB of "standard" DDR4.
I really like OSTEP's chapters on virtual memory if you're interested in reading more[1].