Hacker News Reader

Comment by wtallis | Hacker News Reader

To put some numbers to this:

In 2005, Opterons used a 940-pin socket and had dual-channel DDR, for 6.4GB/s total bandwidth to feed two cores. In 2009, Xeons the LGA1366 socket to provide triple-channel DDR3, 32GB/s for four (later six) cores.

Now, we're up to 4094 pin sockets providing 8 channels of DDR4 for a total of 204.8GB/s of memory bandwidth for 64 cores, which is exactly the same per-core bandwidth we had in 2005.

(And for comparison, a $200 graphics card will also have around 200GB/s of memory bandwidth.)

View on HN

> which is exactly the same per-core bandwidth we had in 2005.

Modern CPU cores are way faster than ones in 15 years old CPUs.

That 2015 processor has AMD K8 cores. I wasn’t able to find exact info, but slightly newer AMD K10 can do 8 single-precision FLOPs/cycle, at 2.8 GHz.

Modern Zen2 cores can do 32 FLOPs/cycle at approximately same frequency, i.e. per-core performance improved by a factor of 4.

Both numbers are theoretical limits only achieved with very specific workloads and heavy use of manual vectorization. Still, real-life general-purpose code scales quite similarly between them. CPU benchmark says their single-threaded performance improved by a factor of 3.5.

The larger and more complex cache hierarchies will go quite far in compensating for the increased per-core performance.

Unrelated question, what about RAM latency? Did it significantly improve since 2005?

So at the consumer level, DDR2[popular standard c. 2005] could be clocked up to 266 MHz whereas DDR4[popular now] clocks in at 1600MHz. So a little more than a 6x speedup in transaction times. Unlike processors, whose features make it quite difficult to compare against each other, DRAM has stayed relatively straightforward: its still just a bunch of densely packed flip-flops connected to a memory controller that offers bank-strided column/row access.

As you can imagine, processor performance has grown a lot more than 6x over the years. The amount of cores in a package alone has increased more than 6x! So how do we get around this?

1) Larger caches: the most efficient optimization to any slow task is to simply not perform said task. Having a larger L3 cache means sending requesting fewer pages from DRAM.

2) “Hiding” latency with bandwidth: if I have to incur a (relatively) large time penalty tp go to DRAM, I better make sure to make it worth my time. Instead of always getting data in 4KiB pages, often times modern processors support “superpages” all the way up to the gigabytes. If you’re going to go drive out to the store, it makes sense to buy groceries for the week instead of just the candy bar you’re craving right now, right? That way when you actually have to cook the work has already been done.

I am probably missing some nuance here, especially when it comes to memory controllers and putting caches on DRAM, but this is the gist.

But frequency of RAM and latency is not the same. DDR2-800 can have something like 4-4-4-12 timings, while DDR4-2666 can have something like 15-17-17-35 timings. So their result latency will be around the same! And those numbers I just googled, I did not made them up.

DDR2 vs DDR4 has effectively zero impact on latency. Due to the physical distances between RAM modules and the CPU, that’s been stuck within a narrow range for decades. https://en.wikipedia.org/wiki/CAS_latency

The specific implication details make a difference, but nothing is going to provide a 4x improvement in latency so other considerations take precedence. The real change is ever growing cache sizes allowing CPU’s to minimize random RAM reads.

Is there a reason why they don't package a couple GB of DRAM in there?

Actually yes, the cache hit ratio is not linear with cache size so you get diminishing returns. http://www.dataram.com/blog/?p=112. On top of this, once a CPU core is operating at say 80% of the time the maximum possible speed up from more cache is 20%.

Further, every layer of cache adds latency and cost as you need to check it before going to main memory. In the 486 and early pentium days External Cache was common, but it’s no longer useful.

I had something more like a swap than a cache in mind. Expose it as additional physical address space and let the OS decide which pages live there vs main memory.

Yes. The very same reason why they waited years before they eventually moved the memory controller on to the CPU die. It's expensive m2 real estate and eats a big chunk of the waffer. Furthermore, there is diminishing returns regarding all kinds of cache. Most of the recent performance increases are attributed to the very dangerous practice called branche prediction.

CAS has nothing to do with "distances between RAM modules and the CPU".

The memory controller referenced in CAS is in the CPU and the ram modules are part of those ram sticks so it’s not the only component of CAS latency but it is a big one.

“For a completely unknown memory access (AKA Random access), the relevant latency is the time to close any open row, plus the time to open the desired row, followed by the CAS latency to read data from it. Due to spatial locality, however, it is common to access several words in the same row. In this case, the CAS latency alone determines the elapsed time.”

No, distance from memory controller is NOT a component of CAS. CAS is the time it takes for row amps to stabilize their levels after being connected to selected memory cells (capacitors). CAS is counted in clock cycles. Have you ever heard of a motherboard where you would have to add/subtract CAS cycles from memory module specification because of memory slot layout? ;-)

btw 10cm of track on pcb is about 2 clock cycles at 3GHz, ~0.6ns. Now you will love this one - in the last 20 years we went from ~15ns CAS down to ... 7ns, twice as fast! Even better - we got down to 7ns in 2006 https://www.newegg.com/g-skill-2gb-240-pin-ddr2-sdram/p/N82E... and stayed there for the last 14 years.

What we got instead is higher density and susceptibility to rowhammer.

CAS is the full round trip time, the controller does not care about individual components of the latency just the total latency. End users just see the CAS number, but what it’s repressing under the hood is not simply what’s going on inside a DIMM, which is why the numbers get higher with higher memory clock speeds.

Further, MB traces are just part of the story. It’s the physical distance traveled from a CPU’s memory controller to the furthest physical cell location on a chip which is significantly more than 10cm and it’s a round trip timing. Even just the physical DIMM’s are 13 CM wide.

A more accurate ~25 cm each way = 50 cm total ~= 3ns out of 6.56ns timing for a DDR3-2133 & CAS 7 ~47%. CAS 18 DDR4-4600 is 7.82 ns or ~40%.

PS: As to progress, DDR2-1066 at CAS 4 was 7.5n where CAS 18 DDR4-4600 is 7.82 ns that’s some progress.

2005 Intel server memory controller datasheet states up to 800mils for die to BGA ball distance (huge chip), thats 2cm. Newer stuff is hard to dig up. Afaik JEDEC DDR3/4 layout guidelines state no more than 4.5 inches to DIMM socket, 5 inches ball to ball. Max connector to ball length for DDR4 UDIMM is ~3cm. All in all max ~17cm.

> Instead of always getting data in 4KiB pages, often times modern processors support “superpages” all the way up to the gigabytes.

Page size is irrelevant for purposes of getting data from RAM to CPU / cache. This is done by cacheline size, which is usually 64 B. Higher page size helps with better TLB efficiency and also for disk to RAM copy (if that is strictly page-based).

Update: I was wrong about pages, but the fundamental notion that we're moving towards "solving" latency issues with bandwidth hasn't changed. Paging strategies don't change CAS timing (which is what I should have replied with), but your CAS timings are only half of the equation, hence my emphasis on bandwidth.

Nope, you're at the mercy of caches to keep the beast fed.

Memory latency was in the 10 ns ballpark 20 years ago and still is. This is unlikely to change unless memory is migrated to an entirely different physical storage mechanism.

It takes light 1 ns to travel 30 cm (i.e. 1 standard body part / foot). So getting much faster must mean getting closer. I reckon it is only feasible if the RAM crawls into the CPU.

>I reckon it is only feasible if the RAM crawls into the CPU. reply

And God said, "Let there be Cache." And there was Cache.

I think the proposal is to solder an entire stick of ram onto the CPU.

Or CPU onto the RAM stick...

https://www.nextplatform.com/2017/02/23/promises-challenges-...

From A Certain Point Of View™, this is what the RPi and SO-PINE compute modules are, except unfortunately they're no longer usable as memory modules per se.

So a level 4 cache?

let's have memory sockets behind the cpu motherboard area !

At this point I wonder why arent CPU vendors experimenting with memory compression. Works great in GPUs. This would have a chance of cutting latency if decompressing a cache line is faster than loading whole thing from ram.

Texture compression works because GPUs are bandwidth-limited in regards to memory. It doesn't really help you when you are latency-limited, because the latency-to-first-byte is the same or slightly larger for compression.

You arent always interested in first byte, besides CPUs operate on whole cache lines. Doesnt matter where the byte is, you will have to wait whole load anyway. We already have memory encryption.

"AMD mentioned a latency increase of 7-8 ns when memory encryption is enabled, which results in a 1.5% performance hit in SPECInt"

Here we would be compensating with smaller data transfers. Wouldnt have to be an amazing algorithm, even RLE would probably give latency improvements.

Reply on news.ycombinator.com