Hacker News Reader

Comment by hinkley | Hacker News Reader

My understanding is that branch prediction got better in the ‘10s and a bunch of techniques that didn’t work before do now.

View on HN

The modern VM technique looks almost exactly like what the original PIC papers talked about in the 90s. There are some details that are different, but I'm not sure that the details come down to exploiting changes in branch prediction efficiency. I think the things that changed come mostly down to the fact that the original PIC paper was a first stab by a small team whereas modern VMs involve decades of engineering by larger teams (so everything that could get more complex as a consequence of tuning did get more complex).

So, while it's true that microarches changed in a lot of ways, the overall implications for how you build VMs are not so big.

Are you still using a threaded interpreter main loop? That didn't really come around until the mid 90's and I've been hearing for about ten years now that it's not a clear win anymore due to predictors being able to read through two levels of indirection.

The last time I ran the experiment of having a single jump, it was slower than jump-per-opcode-handler.

It's true that predictors are able to see through multiple levels, but a threaded interpreter gives them plus one level, and that ends up mattering as much as it always did.

Threaded dispatch is absolutely worth it. Wizard's interpreter gets anywhere from 10-35% performance improvement from threaded dispatch.

> that branch prediction got better in the ‘10s and a bunch of techniques that didn’t work before do now.

They got better than they had any right to be, but then we found out that Spectre & Meltdown were vulnerabilities rather than optimizations.

For example, a switch based interpreter was fast as a CGOTO one for a brief period between 2012 and 2018, but suddenly got slower again as the CPUs could no longer rely on branch prediction to do prefetching.

While better predictors allow the speculation window to be larger on average, the the real culprit is that large speculation window. Even if the branch predictor weren't very smart, it will still do well on a program with stable, predictable branches, thus allowing a large speculation window to open up. The vulnerability is that some of those branches guard really important things, like not going out-of-bounds of an array. So a Spectre attack, which works by exploiting a mispredicted branch, is a constructive attack where the gadget is tuned for the branch predictor anyway. The other part of an attack, the windowing gadget, just relies on making a really slow input into a branch. Neither of them would be particularly harder with a dumb predictor.

Reply on news.ycombinator.com