This is a bad week for speculative execution.
Speculating within one security domain would not be problematic, because anything such speculation allows to glean is already accessible normally. This is sort of similar to having a single reference to a mutable object, which is also safe.
All the various mitigations have already been very costly (something like a performance loss of 10% on current-gen CPUs?). Getting rid of speculative execution entirely would cost far, far more.
It's very hard to not make that mistake. Think of all the websites running in your browser on your laptop. Isolating all those trust domains is going to be quite costly.