Latency numbers every programmer should know (2012) | Hacker News Reader

Latency numbers every programmer should know (2012)

gist.github.com

224 pointsokket42 comments

Instead of memorizing tables, it would be more valuable to programmers to read how these numbers come to be in the first place. An excellent, and much longer read, is by Ulrich Drepper[1].

[1] http://lwn.net/Articles/250967/

I came across this article the other day for the first time and started to read it. How much still applies to today?

Things haven't changed much since the other day, so probably all of it.

GP probably didn't mean "9 years ago" (the article was published over 9 years ago) when saying "the other day." 9 years is a long time in the tech world. It's a valid question.

Why do you say so? AMD k8 came out 14 years ago and PC architecture is basically unchanged since.

From a quick look through, most of it. Modern x86 micro-architectures still use integrated memory controllers and ~3 level CPU caches.

All the timings are a little faster, the caches a little bigger, and the buses a little wider, but it's still basically the same stuff with different names.

Part 1: 50% applies, RAM controllers now part of the CPU, deprecating half of what’s written, also some CPUs have eDRAM. Part 2: almost completely applies. Part 3: 70% applies, hardware SLAT (Intel EPT / AMD RVI) deprecated what’s written about virtualization. Part 5: 90% applies. The rest of them — don’t know.

P.S. What’s I dislike most about the article, it fails to explain why L3 cache is 10-20 times slower than L1 cache, while they both made from SRAM.

> it fails to explain why L3 cache is 10-20 times slower than L1 cache, while they both made from SRAM.

Why is it? Is it because L3 is usually shared?

Even when the cache line is unshared, L3 is still 10 times slower than L1.

The best explanation I saw is this: https://fgiesen.wordpress.com/2016/08/07/why-do-cpus-have-mu...

I think generally the larger the cache the higher the latency. I think various factor are things like available power budget, some increase in signal propagation delay and more complicated logic needed in a larger search space.

Without having read the article I assume it is something to do with the asseociscativity and size of it?

I think its less important to know the exact numbers since those are a moving target but its more important to understand the orders of magnitude in difference from fastest to slowest and in between.

I browsed the comments hoping there would be a better alternative in the comments. Could not have hoped to find a better article. Thank you, the first two chapters are a great read. Bookmarked for the weekend.

On the date of publication and rapid changes in tech world. It might be a bit older but the generic concepts are still great to learn if you only heard the terms before but don't know the details.

PDF version of said same article.

https://www.akkadia.org/drepper/cpumemory.pdf

Need to supplement this excellent article with Row Hammer information. It doesn't cover clflush. I sent the author a note.

"The original document prints out at over 100 pages." - for reference

Reminds me of the video of Grace Hopper talking about nanoseconds:

https://www.youtube.com/watch?v=JEpsKnWZrJ8

It's also powerful to think of these numbers in the context of complexity. You might not worry about a nanosecond here or there in an O(N) algorithm, but at O(N^3), when N is 100 each nanosecond in the inner loop is a millisecond of wall clock. Blowing the cache in that inner loop is going to hurt

Can confirm. Currently scrambling to finish up code for school, and the additional synchronization involved in false-sharing is killing me.

I agree whole heartedly that these are good numbers to have in your head.

http://www.bitmover.com/lmbench/

measures a bunch of these in a portable way. It tries to give you latency & bandwidth of all the things.

Always always measure.

Better than guessing based on wrong or old information

I'm considering an "Ask HN" on things programmers should know.

There's "things programmers should know" according to HN: https://hn.algolia.com/?query=%22programmers%20should%20know...

And as previously mentioned, 97 Things every programmer should know: collective wisdom from the experts, Kevlin Henney, ed., O'Reilly, 2010

http://www.worldcat.org/title/97-things-every-programmer-sho...

Most of the latter is practices, with a few technical recommendations: DRY, floating-points, IPC, linkers, BTS. Most, however, aren't.

I searched DDG and github for "every programmer." Man, I gotta get busy, that's a lot to know.

Fortunately, there's a book: 97 Things Every Programmer Should Know

https://books.google.com/books/about/97_Things_Every_Program...

http://www.worldcat.org/title/97-things-every-programmer-sho...

http://programmer.97things.oreilly.com/wiki/index.php/97_Thi...

Free Gitbook version @ https://97-things-every-x-should-know.gitbooks.io/97-things-...

Not every programmer needs to know these numbers.

I agree that for most programmers it's more valuable to be able to communicate clearly, both in person and in code with the future reader.

Its not so much about programmers needing to know the numbers.

Its more about programmers needing to know what L1 cache is, the idea that some operations are faster than others, etc.

I know a lot of web dev type people who have no idea how the CPU works, what a register is, what paging or virtual memory are, etc. When you're treating compute resources like they're free and abundant (what web devs like to do these days), then of course you don't care. I just wish those devs did care, because their fancy dev machines blind them to the fact that their theoretically simple web app groans on anything other than an i7 with 8Gb ram. Their fast internet and local servers also seem to make them forget why its bad that first load requires megabytes of js. Sometimes I'd rather browse with my cheap tablet and that nonsense seriously blows.

There's a lot of gluttony in development these days. I wish every developer was required to take a basic OS or assembly course to see just how much is happening between writing js and having it actually execute. To see what it really means to program a computer and not a web browser.

I also wish more devs would take a look at the performance of MS word and the performance of Google Docs and apply a little critical thinking. On my cheap Surface 3 (4gb ram i3) Word loads instantly, sips power, and does everything I'd ever want locally. Docs takes forever, destroys the battery and is slow as molasses in January with both Edge and Chrome.

Yes, every programmer does need to know these numbers and why the numbers are what they are.

Sadly such (willful?) blindness is far from unique to web devs (though much of it seems to originate with web dev these days).

I have personally encountered people that dismiss the whole "lets stuff everything in /usr" issue with claiming that everyone (or at least those they care about) are using lights out management anyways.

I used to think that. But I was wrong. These relative ratios continue up the stack. Blowing them off is just excuse-making that gives us bloated and slow software.

You may be a programmer in a language that doesn't permit you to influence L1 cache performance, for instance, but you better understand the mechanisms involved and how that applies to your language and computational model 5 layers up.

Exactly. It's the same myth as premature optimization. Those people tend to ignore good data design (shrink the structs, share fields, order your fields, prefetch arrays, avoid ptr chasing, no trees but tries, hashes over trees, ...) and rather think in costs by lines or ops. Which is horribly wrong for the last 15 years.

Given that any simple bit or int arithmetic might be 50x faster than accessing a bloated field in a bloated struct, they'll never be able to write performant software, yet understand why more code and more lines are faster.

According to sources like this https://www.cs.utah.edu/~manua/pubs/systor15.pdf NVMe 'disks' give another 5-10X IOPS boost. With M.2 the pricing is pretty decent for many things.

I think programmers or system designers that miss things like new secondary storage technologies are doing it wrong because the orders of magnitude can really affect development time.

This is super applicable, even in web dev. Properly using a clustered index in an RDBMS causes sequential reads instead of random reads.

That knowledge alone can speed up queries orders of magnitude and prevent the need to move to a NoSQL solution.

I think it's far more useful to remember ratios rather than exact numbers.

Ratios change too. PCIe / NVMe are better than SATA.

An important reason we keep using strategies that have been abandoned in the past is because the ratios change over time.

Ubiquity of SSDs brings to a close a time when getting data from another computer is faster than getting it from your own disk drive. A ratio that had previously been true, IIRC, not since a time in the eighties (see the Sprite system, process migration).

For those interested in CPU specific numbers I've found this site [1] to be quite useful.

[1] http://www.7-cpu.com/

I've seen this chart in various forms going around for years. Do the numbers ever go down or is this the same chart from 5 years ago? In which case I would think the numbers would be high.

Assuming ~1GB/sec SSD

Is this from the future?

Not very common at the time but they did exist! Here's a review of one from 2012: http://www.anandtech.com/show/6124/the-intel-ssd-910-review/...

The SATA III bottleneck is still common today, but Mac laptops, at least, have sported SSDs that can do ~1GByte/sec for a couple of revisions now. I think they're PCI Express too.

Intel 750 PCIe/NVME SSDs have 2GB/s sequential reads and 20us random read latency.

My laptop has a pair of NVMe drives that can easily read at that rate.

Reply on news.ycombinator.com