Hacker News Reader

Comment by gpderetta | Hacker News Reader

Unfortunately fetching a new cache line from memory is a globally visible side effect. One of the major benefits of speculation and is that it enables additional memory level parallelism.

Even otherwise strictly in order designs, when aiming for high performance incorporate some form of memory speculation (scout threading, run ahead execution).

View on HN

Right, and CPUs could, in principle, fix this. Eg. they could speculatively fetch the cache line, and only commit it to the cache once they know what branch they took. This still lets you do the slow memory access before you know if you will need it, without causing globally visible side effects.

As soon as you fetch the cache line, every other core will know you have done that via the cache coherency protocol. You have just disclosed an address potentially derived from some sensitive information.

Edit: to be precise they can, for example, see the transition from exclusive to shared.

I suppose this gets into what is consider a "major" overhaul of the architecture. The idea is that you don't actually "fetch" anything until you are no longer speculating. The core that is doing the speculating can pre-fetch the memory, and then commit it to the cache only once it knows that is the correct thing to do. There are still some cache coherence concerns, but nothing that strikes me as insurmountable.

I do not understand your distinction between prefect and fetch. You either have the data, and you need to notify other cores that you do or you don't.

Not notifying other CPUs seems pointless because before committing you would have to do a round-trip to the other core to verify whether the data is still valid making the prefetching pointless.

What you are describing here is high cost, because you have to take care of coherence when the line is committed to the cache, which in your scenario would not be speculative. That would require notifying the other cores and potentially refetching if another core has written to the line between your prefetch and commit.

A more practical solution is probably to have protection boundaries within the cache, like http://people.csail.mit.edu/vlk/dawg-micro18.pdf.

Reply on news.ycombinator.com