It's time to figure out how to make RAM dense, cheap, AND fast, so we can finally eliminate the cache layer from the ridiculous hierarchy of storage technologies.
[1] https://stackoverflow.com/questions/4087280/approximate-cost...
A different way to attack this problem is to make timing attacks harder by disallowing programs from reading the current time or inferring the passage of time by monitoring non-deterministic interactions between threads.
Pure Haskell code (i.e. code that runs outside the IO monad) is an example of an execution environment that conceals timing from the running code.
There are certain kinds of problems that aren't amenable to running in a completely side-effect and non-determinism free sandbox, but maybe for some applications it's a viable solution.
We aren't at full lightspeed latencies ram technologies, but we're closer than you may realize, and it's unlikely we can do substantially better any time soon without revolutionary advances.
[Edit] I realize it would always be slower than on-chip cache, but I thought it would still be at least an OOM faster then conventional DDRx DRAM.