I've been saying since this initially came up that big.LITTLE is the long-term solution for this.
In the grand scheme of things, high-intensity tasks are only infrequently high-security tasks - those two sets of workloads are mostly disjoint. So the long-term solution is to have "fast cores" and "secure cores".
The fast cores can have all the OoO, speculation, all of that good stuff. That's where you run anything that needs to go fast, or anything running "trusted" code. By and large, nobody cares if an ffmpeg process or HPC node might leak data. Databases? You control the queries that are running on them, right? There are some edge cases like video games where leaking data is moderately harmful (could be useful for exploits if you can reliably leak useful data) yet you still want maximum performance, but at the end of the day leaking data at a couple kB/s usually isn't going to be the end of the world especially if the data is rapidly changing.
If the code is untrusted or user-generated, or the data is sufficiently sensitive, then run it on a "secure" core. The "secure" cores have to be in-order, non-speculative, all that crap. Probably non-SMT as that seems to be a bottomless pit of sidechannels as well. But usually, you aren't churning huge workloads in the "secure" situations. You can still have crypto acceleration instructions built into the cores, AVX, whatever, just not speculative. It's probably better to get them fully out of the "normal" cache hierarchy as well.
There are a couple obvious problems here, but much smaller than trying to fix everything for every use-case. In particular web browsers are running untrusted code, and every single website is running 15 mb of shitty javascript code. It sucks but it's basically become an inner platform and you can't trust the code that it's bringing in, so that needs to be permanently isolated on its own secure cores. People will have to start paying attention to the performance of their javascript and optimizing out the real shitty bits.
Another big one is shared hosting environments - VPS environments are a prime target for trying to leak data from other clients on the same core/cache hierarchy, so those either need to be moved to "secure" cores, or switched to a model of renting out a whole core (or moved to a "hard time slice" where when the slice goes active you get the whole core for X seconds, then the processor stops, flushes everything, then switches clients). But VPS could conceivably be moved to "arrays of little cores" (to the extent that they aren't already) and that won't pose much problem for a lot of typical "micro" use-cases as long as every instance doesn't hit the server at once. Maybe for people that need faster than a dedicated "little" core the next increment becomes leasing the whole core, or even the whole complex of cores on that cache hierarchy.
Web application servers (not necessarily databases) are another one, unfortunately, since you can time web requests and use that to "leak" data down different code paths. If it's a directly user-facing service, probably best to get it onto a secure core.
The big task for humans is going to be identifying what stuff is allowable to run on the "fast" cores, and then get the schedulers set up so they understand that some stuff can only run in certain processor domains. It's not insurmountable, it just is going to take some time to plug away at it. Perhaps distribute whitelists, and allow the end-user to manually override it if they're really sure.
But yes I've been saying that too, my suspicion is that basically all of OoO and speculation is fundamentally incompatible with not leaking timing data between processes, and that the harder we tilt at this the more attacks we're going to turn up, it's going to turn into an endless game of whack-a-mole and it's going to eat up all the performance gains that we've spent the last 20 years building on the backs of OoO and speculation.
AMD is quite well-placed for this imo since each CCX basically acts like its own NUCA (non-uniform cache architecture) domain and they just happen to share a memory controller. That's pretty much the design you need to make it work right, just with big and little CCXs instead of only big. They just have to come up with their own little cores. Intel is going to be harder because the classic Sandy Bridge architecture (which is largely unchanged today) has all the cores collectively sharing their last-level cache, and I think that's probably a problem in the long term too. I think Skylake-X still works on the principle of cache being attached to each core and them talking to each other to share it.
AMD and Intel engineers, please make your consulting checks out to 'cash'. Thanks! ;)