You could ask the same question about any class of security bug, so unsurprisingly I'd answer more or less in the same way.
For example, if you're paranoid, make your compiler be conservative, in the same way that you might address buffer overflows by using a language/compiler that inserts bounds checks everywhere.
If you're less paranoid and/or more worried about performance, invest in static analysis tools or languages with augmented type systems. After all, you only have to worry about Spectre variant 1 when handling attacker-controlled data. Tracking type info like this is already done by existing static analysis tools.
Finally, if you're not handling attacker-controlled data at all - which is the case for a lot of performance-sensitive code - you really don't want to (and don't have to) do anything about Spectre variant 1.
By the way, this is really the big difference between the two Spectre variants, and why it's a shame that they fall under the same name. Variant 2 affects all code with indirect jumps/calls, even code that doesn't ever touch attacker-controlled data. That's a huge difference between the variants.
Anyway, the bottom line is that you shouldn't punish the performance of all code over a class of security bugs that a lot of code isn't affected by. Buffer overflows haven't stopped us and shouldn't stop us from writing performance sensitive but security uncritical code in unsafe languages either.