> In your example, it sounds like you mean throughput-optimized. According to the original post, brewing hot water for 10 cups would introduce additional latency.

Nope. CPUs are latency-optimized.

The "1st cup of coffee" always takes the same amount of time in CPU-land. The 2nd-cup of coffee was speculatively made, but never "slowed down the first cup of coffee".

----------

A throughput optimized machine, like GPUs (and strangely enough: hard drives), are willing to slow down the 1st-cup of coffee for better overall throughput.

Hard drives are interesting: if you have the following "reads":

#1: Read location 1

#2: Read location 100

#3: Read location 50

The hard drive will re-arrange the reads into: Read 1, Read 50, Read 100, because the hard-drive head will reach location50 before location100. Remember, hard drives are physically moving their arms to each physical location.

This means that Read100 is "slowed down", its latency got significantly worse. But the three reads all together all completed at the same time.