Unrelated question, what about RAM latency? Did it significantly improve since 2005?
And God said, "Let there be Cache." And there was Cache.
https://www.nextplatform.com/2017/02/23/promises-challenges-...
"AMD mentioned a latency increase of 7-8 ns when memory encryption is enabled, which results in a 1.5% performance hit in SPECInt"
Here we would be compensating with smaller data transfers. Wouldnt have to be an amazing algorithm, even RLE would probably give latency improvements.
As you can imagine, processor performance has grown a lot more than 6x over the years. The amount of cores in a package alone has increased more than 6x! So how do we get around this?
1) Larger caches: the most efficient optimization to any slow task is to simply not perform said task. Having a larger L3 cache means sending requesting fewer pages from DRAM.
2) “Hiding” latency with bandwidth: if I have to incur a (relatively) large time penalty tp go to DRAM, I better make sure to make it worth my time. Instead of always getting data in 4KiB pages, often times modern processors support “superpages” all the way up to the gigabytes. If you’re going to go drive out to the store, it makes sense to buy groceries for the week instead of just the candy bar you’re craving right now, right? That way when you actually have to cook the work has already been done.
I am probably missing some nuance here, especially when it comes to memory controllers and putting caches on DRAM, but this is the gist.
The specific implication details make a difference, but nothing is going to provide a 4x improvement in latency so other considerations take precedence. The real change is ever growing cache sizes allowing CPU’s to minimize random RAM reads.
Further, every layer of cache adds latency and cost as you need to check it before going to main memory. In the 486 and early pentium days External Cache was common, but it’s no longer useful.
“For a completely unknown memory access (AKA Random access), the relevant latency is the time to close any open row, plus the time to open the desired row, followed by the CAS latency to read data from it. Due to spatial locality, however, it is common to access several words in the same row. In this case, the CAS latency alone determines the elapsed time.”
btw 10cm of track on pcb is about 2 clock cycles at 3GHz, ~0.6ns. Now you will love this one - in the last 20 years we went from ~15ns CAS down to ... 7ns, twice as fast! Even better - we got down to 7ns in 2006 https://www.newegg.com/g-skill-2gb-240-pin-ddr2-sdram/p/N82E... and stayed there for the last 14 years.
What we got instead is higher density and susceptibility to rowhammer.
Further, MB traces are just part of the story. It’s the physical distance traveled from a CPU’s memory controller to the furthest physical cell location on a chip which is significantly more than 10cm and it’s a round trip timing. Even just the physical DIMM’s are 13 CM wide.
A more accurate ~25 cm each way = 50 cm total ~= 3ns out of 6.56ns timing for a DDR3-2133 & CAS 7 ~47%. CAS 18 DDR4-4600 is 7.82 ns or ~40%.
PS: As to progress, DDR2-1066 at CAS 4 was 7.5n where CAS 18 DDR4-4600 is 7.82 ns that’s some progress.
Page size is irrelevant for purposes of getting data from RAM to CPU / cache. This is done by cacheline size, which is usually 64 B. Higher page size helps with better TLB efficiency and also for disk to RAM copy (if that is strictly page-based).