To add to this and to quote a friend who has more NDAs in regards to microarchitecture than I can count and thus shall remain nameless: "You can have a fast CPU or a secure CPU: Pick one". Pretty much everything a modern CPU does has side effects that are something that any sufficiently motivated attacker can find a way to use (most likely). While many are core specific (register rename, execution port usage for example), many are not (speculative execution, speculative loads). Side channels are a persnickety thing, and nearly impossible to fully account for.

Can you make a "Secure" CPU? In theory yes, but it won't be fast or as power efficient as it could in theory be. Because the things that allow those things are all possible side channels. This is why in theory the TPM in your machine is for those sorts of things (allegedly, they have their own side channels).

The harder question is "what is enough?" e.g. at what level does it not matter that much anymore? The answer based on the post above this is based on quite a lot of risk analysis and design considerations. These design decisions were the best balance of security and speed given the available information at the time.

Sure, can you build that theoretically perfect secure CPU? Yes. But, if you can't do anything that actually needs security on it because it's so slow; do you care?