Saying a different way: for fork/join or barrier style parallel requests, stragglers set overall latency, and though the probability of any specific response being a straggler may be low, the probability of at least one response being a terrible straggler gets very high at large scales (or large fan-outs).

A couple weeks ago someone brought up High Frequency Trading and while in theory it didn’t tell me anything I didn’t already know, I’ve been chewing on the thesis of the linked article ever since: that the real trick to doing things quickly is to do them consistently. That variance causes far more kinds of practical problems than does average response time.