Comment by wahern | Hacker News Reader

Comment by wahern | Hacker News Reader

Parent Full thread

Modern CPUs are like miniature JIT compilers.[1] They do wondrous things but I guess it shouldn't be surprising that in critical sections static compilation with a more global perspective can often make better decisions.[2] I would think that would be even more true today than during the Pentium II era, or at least as true. Both the state-of-the-art and the resources that can be devoted to such optimizations have continued to progress for both static and JIT compilation.

[1] Someone else please speak up, but the Pentium II seems to be when Intel began leaning on microcode to optimize execution. https://en.wikipedia.org/wiki/Intel_Microcode

[2] Not always, of course. Sometimes local, runtime information is more important, as JIT enthusiasts love to point out. The problem is that the instrumentation to analyze and utilize that information is costly and incurred perpetually, whereas with static compilation it's a one-time deal.[3] As Rust and C++ has proven, many developers are willing to put up with several orders of magnitude more costs up-front than could ever be tolerated in a JIT, especially the JITs inside the CPU.

[3] This is a similar issue to GCs (mark & sweep and reference counting), where memory must be scanned and sometimes even copied many times during runtime whereas in static memory management there's exactly one operation for allocation and one for deallocation. So GC is usually (but, again, not always) slower. Optimizations like bump allocations (which in static compilation is often simply "the stack") that aggregate and amortize some costs don't change the fundamental costs.

The extent to which modern CPUs are “like JIT compilers” is highly overrated by SW people.

Most important instructions map 1:1 to CPU micro-ops, with memory operands (on x86) being mapped to separate load/store micro-ops. There are cases where instructions map to long sequences (e.g. VMENTER) or get combined into one micro-op (e.g. compare-and-branch), but this is far from a JIT compiler.

I also had in mind branch prediction units, and stuff like Intel's TSX. I assume these days the former bleeds into many other parts of the architecture, including the microcode; I'm certain (but can't say authoritatively) that the latter does.

From a wholistic viewpoint I think it's fair to characterize modern CPUs as JIT-like. It's easy to trivialize individual aspects of the whole. You can do the same for software JITs. They're not magic (notwithstanding the breathless claims and river of thesis papers), and often simpler is better, anyhow (e.g. LuaJIT). For obvious reasons the tactics employed by CPUs will tend toward the simpler end of the spectrum.

To maintain the JIT analogy: you've addressed instruction-selection, but what about register allocation (with register renaming)?

An alternative view (imho more accurate) on Tomasulo-like processors is that they essentially execute the data flow graph implied by a sliding window of instructions.

Sort of like a tracing JIT? I'm not trying to equivocate modern CPUs with the JVM, just emphasize the similarities as a way to understand and explain observed behaviors.

Tracing JIT tries to synthesize graph of all distinct execution paths (in practice for some smallish value of “all” at a time), ie. there is no sliding window. In comparison OoO CPUs try to execute as much as possible of instructions they see “at once” (this can mean in parallel or in other order than is encoded by the instruction stream). Doing this in hardware involves looking at some block of memory (eg. L1I Cache line or several of them) and doing what is essentially an more involved variant of peephole optimization pass.

Reply on news.ycombinator.com