It's also worth noting that Facebook is actually pursuing this same approach with Hack (making the language more optimizable). And they've tried the AOT approach, and it turned out to be a huge pain for several reasons.
It's also worth noting that Facebook is actually pursuing this same approach with Hack (making the language more optimizable). And they've tried the AOT approach, and it turned out to be a huge pain for several reasons.
The timing results come from within HHVM. Basically, the test is dispatched within HHVM, isolating just the function call itself. It's then executed 5 times (timed independently) and then the bottom 2 executions are thrown away. See: https://gist.github.com/ircmaxell/8859d7f1bbee7b2f6e16#file-...
So the warm-up period happens. It's not just "benchmark `hhvm file.php`". The benchmark happens inside of HHVM.
So yes, JIT warmup is accounted for 100%. In fact, PHP, HHVM and Recki-CT all execute with the exact same pre-compile steps (parsing and any pre-compiling are ignored in the benchmark).
And their AOT approach that they tried was all-or-nothing. Meaning the entire code base had to be converted. This is a "compile what you want, and run the rest normally". In fact, this is 100% compatible with Hack and HHVM...