Hacker News Reader

Comment by userbinator | Hacker News Reader

Looking at the patches Linus is commenting on: as an Asm programmer, this is absolutely horrible --- basically every indirect call (one instruction) turns into a seven-instruction sequence that will, due to preventing speculation, result in massive slowdowns:

https://lkml.org/lkml/2018/1/3/770

Unlike the KPTI patches, which only affect things on each system call, this happens on every indirect call and probably bloats the code considerably too. I can see why Linus is not happy.

Edit: To give a bit more background, predicting indirect calls and speculating into them is absolutely critical to getting good performance from OOP-ish code which tends to use them a lot (virtual functions in C++, function pointers in C). The Linux kernel is (thankfully?) not very OOP-ish, but it does rely on indirect calls (function pointers) extensively.

View on HN

I'm frankly shocked Linus was anywhere near as restrained as he was upon seeing this monstrosity. Nothing of this sort should ever make it into the kernel without explicit and loudly announced opt-out switches. If anything close to this does make it in, even as an opt-in, this will easily be the biggest performance regression in recent (perhaps all-time?) kernel history.

His restraint and the fact that this stuff is even being considered tells me this is a really really bad bug. I doubt he would be so cordial in any other situation.

Nothing like this should have ever made it into hardware. It's lazy implementation (or performance at the expense of security).

Performance at the expense of security is a compromise made daily in every aspect of IT. The only way to make systems perfectly secure is to make them unusable.

Now, you can certainly argue that this particular tradeoff wasn't worth it, and in hindsight it's certainly a questionable one. However, do note that side channel attacks have been known for decades (since out-of-order CPUs became widespread in the 90s) and until this point have rarely presented a significant threat to most systems.

I'd be more inclined to think this was someone who didn't understand the implications implementing a performance improvement. This bug is clearly deep into unk-unk (i.e. well past the known areas of concern) territory and took a great deal of insight to the fundamentals of speculative execution to discover.

I missed that, actually. But you are right, it's not just the crucial bits of inline assembly they are patching here, they expect your kernel to be built with their special version of GCC (yet unreleased) that mutilates every single freaking indirect jmp/call into what could cost more than a virtual function call..

We use a new compiler option -mindirect-branch=thunk-extern

I missed that it was indirect jumps too, which means switch statements also get affected. That's even worse.

Here is an example with the very common programming patterns that result in indirect jumps/calls:

https://godbolt.org/g/LPycpY

Every single call rax and jump rax that you see on the right will be replaced with the following:

    push rax
    jmp .indirect_thunk

And indirect_thunk does an lfence and a call to another function that gets rax from the stack and jumps there in a very roundabout way that interacts terribly with CPU prediction. I can see why they somehow didn't get around to benchmarking this yet, because they are not going to like the results. And it doesn't end with the kernel. Every JIT is going to need the treatment.

That has to be crazy slow. They probably haven’t published benchmarks yet because they’re still waiting for it to finish running!

Interacting terribly with CPU branch prediction is the whole point -- Spectre attacks work by messing with the branch predictor, to misdirect speculative execution onto paths that reveal information useful to the attacker. So, the point of this whole crazy dance is to effectively disable the branch predictor.

Of course. But it needs spelling out, because branch prediction is essential to good performance and why todays CPUs can be much faster despite not reaching higher clocks. It's much more than just the cost of a few more instructions.

Apparently the perf impact isn't so bad outside of microbenchmarks actually. 10% for real world programs ... as long as you use profile guided optimisation to devirt as many calls as possible!

https://support.google.com/faqs/answer/7625886

http://webcache.googleusercontent.com/search?q=cache%3Ahttps...

(some websites with info on this are DoSd and down right now, including the Arm website and LLVM code review site so switched to a cache link)

Pretty painful for C++ but fixable with build changes. JVM users and other JIT-compiled languages will probably be fixable without any serious impact, as HotSpot and other advanced runtimes devirtualise calls (convert indirect to direct) as much as they can anyway, meaning most calls won't be indirect when finished and once the code has warmed up.

The above example compiled with optimizations and with `indirect_jump` altered so the compiler actually emits a jump table instead of an array lookup:

https://godbolt.org/g/MPs43x

Clang has a new code generation option, too.

* https://reviews.llvm.org/D41723

* https://news.ycombinator.com/item?id=16070050

* https://news.ycombinator.com/item?id=16070356

Isn't that patch for the bug that's present on basically every CPU, Intel, AMD, ARM, etc., though? I thought the Intel-specific bug was fixed by "blank out the TLB every sysret".

Different bug, AFAICT.

Reply on news.ycombinator.com