Latency numbers every programmer should know
samwho.dev
samwho.dev
L1 cache reference = 1ns
Branch mispredict = 3ns
L2 cache reference = 4ns
Mutex lock/unlock = 17ns
Send 1K bytes over 1 Gbps network = 44ns
Main memory reference = 100ns
Compress 1K bytes with Zippy = 2us
Read 1 MB sequentially from memory = 3us
Read 4K randomly from SSD = 16us
Read 1 MB sequentially from SSD = 49us
Round trip within same datacenter = 500us
Read 1 MB sequentially from disk = 825us
Disk seek = 2ms
Send packet CA->Netherlands->CA = 150ms
Can we discuss the actual material now.
1. The vertical text is difficult to read despite its size, because it's vertical.
2. When we click on it a large part of the text disappears below the bottom margin of the page.
Problem number 1 is not so bad but the combination with 2 kills the UX. The text in the clicked bar should appear somewhere on screen, horizontally.
Edit: if anybody like me wonders what's Zippy its a C++ compression library from Google. It's called Snappy now [1]
2. I feel a big part of the problem here is that it's not obvious how to get it back once it's gone. I could certainly try making the text visible after the bar is gone.
I'd already seen cost model numbers like these before, but this interactive visualization still seemed to obscure the information as I was taking a first look.
I wonder whether it would be more useful adapted to a visualization/calculator for specific numbers, maybe for multiple operations in an algorithm, and the alternatives for implementing each? (And the click-to-scale is for selecting N for each operation, and maybe somehow constants?)
That’s why we have graphs and charts in the first place.
And that scrolling horizontally would pan me through the content.
Browsing on mobile, I should clarify.
But I’ll add that I also got the hang of scrolling back “in” fairly quickly. After I had zoomed out a couple times, then finally stopped to read the instructions.
The 1Gb network latency figure on this post is complete nonsense (I left another comment about this further down); looking at the source data it’s clear that this is because this isn’t based on a 1Gb network, but rather a “commodity NIC” with this model, and the quoted figure is for a 200Gb network:
function getNICTransmissionDelay(payloadBytes) {
// NIC bandwidth doubles every 2 years
// [source: http://ampcamp.berkeley.edu/wp-content/uploads/2012/06/Ion-stoica-amp-camp-21012-warehouse-scale-computing-intro-final.pdf]
// TODO: should really be a step function
// 1Gb/s = 125MB/s = 125*10^6 B/s in 2003
// 125*10^6 = a*b^x
// b = 2^(1/2)
// -> a = 125*10^6 / 2^(2003.5)
var a = 125 * Math.pow(10,6) / Math.pow(2,shift(2003) * 0.5);
var b = Math.pow(2, 1.0/2);
var bw = a * Math.pow(b, shift(year));
// B/s * s/ns = B/ns
var ns = payloadBytes / (bw / Math.pow(10,9));
return ns;
}
In other words they're not saying "if you send only 1K of data it will take this long". They're saying "if you send 1 GB, then the total time divided by 1 million is this much".
This is a transcription error from the source data, which as it turns out is based on a rough exponential model rather than real data, but first let’s consider the original claim:
If there’s a buffer on the send side, then assuming the buffer has enough space, the send is fire and forget, and costs a 1KiB memcpy regardless of the link speed.
If there’s no buffer, or the buffer is full, then you will need to wait the entire serialisation delay, which is orders of magnitude higher than 44ns.
One might further make assumptions on the packet size and arrival rate distributions, and compute an expected wait time, but otherwise the default assumption for a figure like this would be to assume the link is saturated, and the sender has to wait the whole serialisation delay.
> They're saying "if you send 1 GB, then the total time divided by 1 million is this much".
This would take ~8s to serialise, neglecting L1 overheads, dividing that by 1MM gives you 8us (my ~10us figure above), which is ~200x higher than 44ns.
Looking at the source data [0], it says “commodity network”, not 1Gb, so based on the presented data, they must be talking about a 200Gb network, which is increasingly common (although rare outside of very serious data centres), not a 1Gb network like the post claims.
Interestingly the source data quotes an even smaller number of 11ns when first loaded, which jumps back to 44ns if you change the year away from 2020 (the default when it loads) and back again.
That implies 800Gb: there is an 800GbE spec (802.3df), but it’s very recent, and probably still too specialised/niche to be considered “commodity”.
Digging further, we see that the source data is computed based models that show various bandwidths growing exponentially over time, not based on a any real data, so these data are extremely rough, given these are real figures that can actually be measured:
function getNICTransmissionDelay(payloadBytes) {
// NIC bandwidth doubles every 2 years
// [source: http://ampcamp.berkeley.edu/wp-content/uploads/2012/06/Ion-stoica-amp-camp-21012-warehouse-scale-computing-intro-final.pdf]
// TODO: should really be a step function
// 1Gb/s = 125MB/s = 125*10^6 B/s in 2003
// 125*10^6 = a*b^x
// b = 2^(1/2)
// -> a = 125*10^6 / 2^(2003.5)
var a = 125 * Math.pow(10,6) / Math.pow(2,shift(2003) * 0.5);
var b = Math.pow(2, 1.0/2);
var bw = a * Math.pow(b, shift(year));
// B/s * s/ns = B/ns
var ns = payloadBytes / (bw / Math.pow(10,9));
return ns;
}
Doubt.
40ms - average human thinks the operation is instant.
15s - user gets frustrated and closes your app or website.
I think it's really interesting and instructional to think about why the UX feels so bad. My ideas are:
- The page has one main job: presenting latency numbers to the viewer.
- This job is easy enough. There are many ways to get this done. So people expect the main job to be done at least as good as with these other ways.
- I hypothesize that the page prioritizes other jobs before the main job. It tries to make finding the relationship between those numbers fun to detect. * Users are foremost interested in the main job, but this main job is done poorly because you don't see all latency numbers in one view (maybe after clicking a few times at the right places, but for such an easy task this is way too much work)
- It's very difficult to grasp the mental model of the UI just aby using it. You click somewhere and things happen. Even now that I have used it for a few minutes, I have no idea what it does or is supposed to do. I found it very interesting how much it frustrated my that repeated clicks are not idempotent and made the UI "diverge". It makes you somehow feel lost and worry about breaking things.
- The user must read the help text. But users don't do this. At least I didn't until I was very frustrated. Then this help text changes. And changes again. I don't want to learn a new application only to read a simple list of numbers.
These are my main points, I think. To me, it was very interesting. Thanks for that, samwho. and kudos for sharing this publically :-)
I'm in the middle of writing up a self-reflective post about this and I just wrote the following:
"Ultimately, the way I'm presenting the data is egregious and unnecessary. I can see why people are annoyed about it. The extra visuals and interactions get in the way of what's being shown, they don't enhance it. Tapping around feels fun to me, but it isn't helping people understand. This experiment prioritised form way more than it prioritised function."
We've come to some of the same conclusions, though you in more detail than me. The idea about clicks not being idempotent wasn't something I ever noticed, but now you've said it I can't not.
If you're willing, I'd love to connect with you 1:1 and talk a bit more about this. My contact details are on my homepage.
Sometimes, the inconvenience of a linear scale is the point.
Pages that I think use this technique to really good effect:
https://xkcd.com/1732/ https://mkorostoff.github.io/1-pixel-wealth/
Latency numbers every programmer should know
L1 cache reference ......................... 0.5 ns
Branch mispredict ............................ 5 ns
L2 cache reference ........................... 7 ns
Mutex lock/unlock ........................... 25 ns
Main memory reference ...................... 100 ns
Syscall on Intel 5150 ...................... 105 ns
Compress 1K bytes with Zippy ............. 3,000 ns = 3 µs
Context switch on Intel 5150 ............. 4,300 ns = 4 µs
Send 2K bytes over 1 Gbps network ....... 20,000 ns = 20 µs
SSD random read ........................ 150,000 ns = 150 µs
Read 1 MB sequentially from memory ..... 250,000 ns = 250 µs
Round trip within same datacenter ...... 500,000 ns = 0.5 ms
Read 1 MB sequentially from SSD* ..... 1,000,000 ns = 1 ms
Disk seek ........................... 10,000,000 ns = 10 ms
Read 1 MB sequentially from disk .... 20,000,000 ns = 20 ms
Send packet CA->Netherlands->CA .... 150,000,000 ns = 150 ms
Assuming ~1GB/sec SSD
I've been a developer for embedded systems in the telecom industry for nearly two decades now, and I had never met anyone using something else than "cycles" or "symbols" until today... Except obviously for the mean RTT US<->EU.
On big computers, cycles are squishy (HT, multicore, variable clock frequency, so many clock domains) and not what we're dealing with.
If we're making an architectural choice between local storage and the network, we need to be able to make an apples to apples comparison.
I think it's great this resource is out there, because the tradeoffs have changed. "RAM is the new disk", etc.
Because whether something is 5x slower or 5000x slower matters. Is it better to wait for 10 IOs, random access memory 10000x, or do a network transaction? We can figure out the cost of the memory/memory bandwidth, etc, but we also need to consider latency.
I've done plenty of work counting cycles; but it's a lot harder and less meaningful now. Too many of the things here happen in different clock domains. While it was a weekly way to look at problems for me a couple of decades ago, now I employ it for far less: perhaps once a year.
> Calculation based on cycles are totally accurate whether the number of cores...
No, they're not, because cores contend for resources. We contend for resources within a core (hyperthreading, L1 cache). We contend for resources within the package (L2+ cache lines and thermal management). And we contend for memory buses, I/O, and networks. These things can sometimes happen in parallel with other work, and sometimes we have to block for them, and often this is nondeterministic. In turn, the cycle counts for doing anything within the larger system are really nondeterministic.
Counting cycles works great to determine execution time on a small embedded system or a 1980s-1990s computer, or for a trivial single threaded loop running by itself on a 2020s computer. But most of the time now we need to think account for how much of some other scarce resource we're using (cache, memory bandwidth, network bandwidth, a lock, power dissipated in the package, etc), and think about how various kinds of latencies measured in different clock domains compose.
Things like Cortex-M-- stuff's deterministic. Sure, we might have caches on the high end (M55/88), and contention for resources with DMA, but we can reason about them pretty well.
A few years ago I was generating NTSC overlay video waveforms with SPI from a cortex-M4 while controlling flight dynamics and radio communications on the same processor. RMS Jitter on the important tasks was ~20 nanoseconds-- 3-4 cycles, about a factor of 100x better than the requirement.
But I guess you're right: you could also consider something like a dual-core Cortex-A57 quite small, where all the above complaints are true.
It is also a bit uncomfortable to read 90° text. It's fun to click the bars and play with the UI, but not to actually read what they say. It's a nice visualization, but it suffers from form over function! I can't comfortably use it to learn about the numbers I should know :(
Are you reading on a landscape tablet? I know the sizes of stuff are wrong on that form factor. Desktop and mobile shouldn't have the first couple of bars obscured.
The sideways text is meant to be a subtle nod to the fact the page scrolls sideways, but I agree it's not as nice to read as it would be were the text the right way around.
I am on a desktop with a huge monitor in ultra high res. It is pretty bad.
>> The sideways text is meant to be a subtle nod to the fact the page scrolls sideways, but I agree it's not as nice to read as it would be were the text the right way around.
Then the subtle nod is lost on me... why not turn the text when I click, or have hover text, or make the whole page rotated 90 degrees?
Like the original response, it was fun for one second, then I was like I can not read this stuff, or its painful.
It is big but unreadable in 4k 32inch screen.
- Peter Norvig (original (?)) - http://norvig.com/21-days.html#answers
- Jeff Dean (slides) - https://www.cs.cornell.edu/projects/ladis2009/talks/dean-key...
- Colin Scott - https://github.com/colin-scott/interactive_latencies?tab=rea...
- this post
> Hopper became known for her nanoseconds visual aid. People (such as generals and admirals) used to ask her why satellite communication took so long. She started handing out pieces of wire that were just under one foot long—11.8 inches (30 cm)—the distance that light travels in one nanosecond. She gave these pieces of wire the metonym "nanoseconds." She was careful to tell her audience that the length of her nanoseconds was actually the maximum distance the signals would travel in a vacuum, and that signals would travel more slowly through the actual wires that were her teaching aids. Later she used the same pieces of wire to illustrate why computers had to be small to be fast. At many of her talks and visits, she handed out "nanoseconds" to everyone in the audience, contrasting them with a coil of wire 984 feet (300 meters) long, representing a microsecond. Later, while giving these lectures while working for DEC, she passed out packets of pepper, calling the individual grains of ground pepper picoseconds.
It felt weird that a gaming computer in a datacenter could be "faster" than a computer on my network, but one frame takes ~16ms to render, bandwidth is big enough to stream, network latency might only be another ~frame, and suddenly the image is on my machine within 2 or 3 frames. However there were unexpectedly slow parts! The controller actually ran over WiFi directly, so that inputs went straight to the server rather than via Bluetooth, comparing with Xbox Cloud on a Bluetooth controller, this made a huge difference, but that makes sense because Bluetooth's latency might be 1-2 frames itself. It's counterintuitive to me that the latency from my controller to my computer, less than 1m, might be higher than the latency from my computer, to my router, to my ISP, to Google's DC, and to a server. Similarly, the latency on HDMI from a computer to my TV is in the same ballpark of a few frames because of all the processing my cheap TV does to look good.
I’d never have considered adding WiFi to the controller to _reduce_ latency, that’s absolutely wild. Thanks for sharing!
Just open your site on mobile and imagine that you don't know the dataset by heart.
Can you glance at the values, can you easily compare different values? When you scroll half a screen to the right, are you completely lost? I know I am. When you select the largest or smallest item, what do you see? What if you then scroll to the other end of the spectrum? Can you read what the smallest item stands for? L1 C something? Can you change the scale so that you can improve what you see? Does scrolling up and down behave intuitively?
All in all, it's just impossible to extract useful information and context using this design. It looks great, you could post it on Behance and it will get positive feedback, but when someone actually wants to use it to discover what the data says, it's a very frustrating user experience.
Seems highly unlikely for a wide variety of reasons.
1) If a technology didn’t exist. I’d make the bar black / grey. The pedants will hound you to death
2) A means to see the trend of a given feature on log plot if necessary. Eg Memory access 1980-2030 is interesting.
3) An info icon on the bar (i in a circle) to get details about the measurement. Disks seek for example is pegged at 1ms because it’s where mechanical disks have limits. If so is it track to track or full stroke seek ?
It's neat. OP, have you considered adding a toggle to switch to logarithmic scale? You could retain the time travel feature, but also show all values simultaneously.
I hadn’t, but it would be very easy to add.
Can’t stress enough how grateful I am to Colin Scott for doing the work and open sourcing the calculations for the latencies over time.
How much work was this and what are the trickiest parts? I see a lot of discussion of different platforms & browsers, which has always been surprisingly tough.
FWIW, I think you’re close to a nice UX and it wouldn’t take much to eliminate most complaints, though I’m speculating, and I know (from experience developing UIs) that everyone’s got different expectations and opinions about their UIs so it can be hard to find the maxima…
I don’t mind the sideways text much, but it would be cool if the bars were narrower so they all fit on-screen at once. (I’m on iPad btw.) It would be nice if the text didn’t move with the bar, but always stayed fixed and visible while the bar changed sizes.
It would be nice if the credit dialog could be minimized/toggled. Or maybe positioned as a title banner, and the data didn’t overlap? I don’t know why but overlapping things give me anxiety.
At first I didn’t understand how the bigger/smaller controls on a data bar worked, I thought it was a toggle, so I thought I had a hard time getting the L1 number to come back up, and didn’t realize I needed to click further up near the title overlay. A toggle might be more intuitive? (i.e. click above bar to make it scale to top of screen, click on bar to set scale so that the bar to the right is exactly top of screen…) Might be neat if the bars were draggable - are they already?
You should disable pointer events / text selection etc.
I'm on Android (Samsung Galaxy S23+)
Why can I tap the largest bar so much that it can become as small as others in the end?
Why can't bars just auto-resize when I scroll the page horizontally?
You're the designer here, so it's up to you to shape user experience to deliver your point. Instead you allow users to ruin it for themselves and don't even try to control the narrative of the page. Sorry, don't know what you wanted to do here but I don't think it's working.
A simple ASCII blog post would have been better
Apologies for the directness, but your definition of working doesn’t align with others then. If you mean “working as intended” where the intention is a hard to read and clumsy ux, then yep it’s working. As a proposal to how to make it less clumsy and easier to read, maybe consider having the text not inside the bars?
Do you have any other feedback? What is it specifically about the UX that you find clumsy? I'm still quite new to this sort of thing and do want to improve.
A typical design pattern on mobile is that if information is obscured for some reason, you click on it to expand it. Consider a drop-down text box on a blog: there's a little arrow and cut-off text with an ellipsis (...). When you click on the arrow, the cut-off text expands to fill the screen and allows you to read the rest of it. In contrast to what other users have said, this doesn't need to be idempotent. Tapping again hides the box.
To apply this design pattern to your site, simply make it so that tapping anywhere on a bar brings the UI to a known state, eg with the bar in the center with the text at a readable scale. This would work either horizontally or vertically.
Benefits:
- Your idea of the UI rescaling is preserved, and you can preserve the animations between states. I think the "rescaling bars" idea is fun.
- Cause and effect is preserved. If I want to read the text on the bar, I should not have to click on some arbitrary point above the current location of the bar.
- Further, the user does not have to hunt for the correct spot to click on a bar to make the text visible. Instead, clicking on a bar immediately and always makes all the information on that bar fully visible, by design.
I think this tweak would significantly improve the experience of interacting with the website.
A simple ASCII blog post would have been better
And I probably not the only one that took a few minutes to realize the number referred to the year.
It has given me lots to think about and learn from.
New UI is just a cool animation, but it is unreadable on a 4K 32-inch screen.
Old UI is intuitive and doesn't require 10 pages of instructions and you can actually see numbers and labels all the time.
It is impossible to see times and text labels at the same time.
I am not going to remember what that 100ms was when I am changing years.
When you say it’s impossible to see times and text labels at the same time, are you aware you can move each bar to any height on the screen? They move to the height of the cursor when you click. It’s extremely unobvious, I know, I’m sorry.
I have to make bar smaller (hide text label) to see time, but then I cannot see the text label.
I want to compare something in middle (memory read) and last bar (CA->Ned) at the same time.
Instead, just show an isometric graph with dates moving right-down, and each type, right-up, if that makes sense. Then, a single static 3d-ish image would show the different latencies and how they've changed over the years. No need for all the interactive gadgetry, that adds nothing, and in fact just obscures the big picture.
https://camo.githubusercontent.com/6e7f6707a2532cca1a5bf4ffb...
Unless using log scales, you might need some way to adjust the scales so that everything fits - maybe click on a 'column', or even an individual bar, to normalize it, and everything else scales relative, somewhat like your site.
Love the site.