It seems to me that almost all optimisations (and especially the accumulation of optimisations that we want to get more performance), that all optimisations are going to be vulnerable in some way to timing attacks.

After all an optimisation is designed to alter the timing. If all actions must have the same timing, optimisations fail.

Do we simply offload any sesnitive processing to specialised chips? The silicon wafer seems to be developing vulnerablities like a network.