See: https://www.phoronix.com/scan.php?page=article&item=3-years-...
These sorts of attacks will get more sophisticated.
Side channels through miss-speculation also have a fun property of being virtually undetectable since the problematic code never actually executes. This is attractive for very powerful actors who might want to spend the extra effort for the covert attack even if there are simpler exploits to actually launch.
Analysis of such vectors is important but the threat is limited. I still favor running encryption in software and find hardware support often quite dubious because you can never be sure here while any runtime attack can just as well be mitigated on a higher level. Doesn't mean it is more secure out of the box but security is about trust as well.