It can mostly be fixed by making sure aborted speculative execution has no side effects.

I think this is doable by adding a buffer to store cache entries evicted by speculative execution, so they can be put back, and a log of new cache entries, so they can be removed

Also, the abort process itself needs to take a constant amount of time out of the critical path to avoid leaking info via timing (the timing then only depends on how fast the CPU determines it speculated wrong, but that doesn't depend on the speculated code).

EDIT: Hyperthreading also needs to be handled somehow, which other than disabling it might require adding a per-thread "L0" cache and using that exclusively for speculatively executed code.

There is still the problem of theoretical plain timing attacks on normally executed code though, but that's unrelated.

Making sure that the indirect branch predictor only makes predictions when the source address fully matches would be nice as well (although theoretically not required if there are no side effects).