This would allow keeping the performance of speculative execution in the vast majority of cases, while also fixing the kind of potentially leaking double-indirections that Spectre variant 1 can exploit.
Spectre variant 2 needs to be fixed by isolation of the branch prediction structures, i.e. tagging the BTB and others with the PCID and privilege level.
That's sadly not enough.
There are also potential speculation-based side-channels that are unrelated to cache: timing the idiv operation whose latency "depends on number of significant bits in absolute value of dividend." Furthermore there are many ways to measure contention on the execution ports which can leak information about the speculative execution.
You mention idiv, which is an interesting example. Is it actually observable? Without hyper-threading, almost certainly not, because the execution unit reservations should be dropped as soon as the speculated branch is resolved. With hyper-threading? Perhaps, but it shouldn't be too hard to fix if a non-speculated thread always takes precedence over a speculated one, which makes sense anyway.
In both cases there may be leaks in practice, but those should be fixable relatively easily in hardware, and we've already established that hardware fixes are required anyway.
Another case to think about are dependent conditional branches, since those could affect the instruction cache.
It does seem like a good idea to have a strict barrier instruction against speculative execution just to have a fallback mitigation.