A lot of poor performance in the wild comes from not batching operations that can be batched, thus increasing latency unnecessarily.
Utilization of the CPU's cache line is just one instance of this problem.
For example, imagine trying to parse a text file by literally reading one character at a time from the OS (literally issuing a 'read' command per character) vs reading the entire file upfront and then operating on the text in memory.
Even the if we assume OS does caching for disk data to minimize the actual number of reads from disk, you will still incurs at least an unnecessary cost from issuing all the sys calls (one per character).
The same can be said about, for example, issuing sql commands, and other things.