I suppose this gets into what is consider a "major" overhaul of the architecture. The idea is that you don't actually "fetch" anything until you are no longer speculating. The core that is doing the speculating can pre-fetch the memory, and then commit it to the cache only once it knows that is the correct thing to do. There are still some cache coherence concerns, but nothing that strikes me as insurmountable.