LuaJIT Remake: An ongoing attempt to re-engineer LuaJIT from scratch
github.com
github.com
They’ve loosened up on interpreters, which is nice, but there is no way to make a JIT.
Which cache the generated code between runs, unless the original code changes, avoid each application shipping their own mini-JIT toolchain, and on Android's case AOT compile the application using the PGO data gathered by the JIT (also shared via PlayStore to other devices), when the device is idle and charging.
Sure, that’s an interesting thing you can do and get a Lua JIT, but I don’t see how that is going to be close to the original, tracing LuaJIT, where much of the simplicity (AFAICT) comes from how it creatively punts on several hard backend problems: optimizing in the presence of complex control flow (an linear sequence possibly followed by a loop, that’s all you get on the lowest level), tricky monomorphization decisions (the types you traced are the types you codegen for), eliminating allocations (exiled to trace exits as much as possible), etc.
Is there a “Heart of LuaJIT” style implementation somewhere? Something that’s like a thousand lines but conveys the core ideas.
The magic of making Lua so fast is seriously underrated. I’m fascinated why Lua in particular seems to be so amenable to speedups vs, say, Python bytecode or even Elisp bytecode.
https://www.freelists.org/post/luajit/How-does-LuaJITs-trace...
Never did program much in Lua but really enjoyed the parts where they describe the virtual machine and instruction set.
EDIT:
Link to the paper I mean https://www.lua.org/doc/jucs05.pdf
For a layman like myself this is very approachable. Again I cannot think of another VM language that has anything this small and well documented.
Elisp bytecode now has a compiler using libgccjit, in Emacs 28.
LuaJIT simply did a lot of heavy lifting, implementing a full blown compiler with register allocation and all of that. Lua wasn't special that way. Rather, the Python devs didn't care that much about speed, and Python has the baked in notion that Python programs are a tree of nested dictionaries. Thus, there is only so much performance that could be wrung out of PyPy. But it still helps a lot.
Python 3 was of course a tragedy. If they were going to break stuff at all, they could have made some of that dynamicity optional (few programs use it). Then PyPy could have been the reference implementation and it would be crazy fast, eliminate the GIL, and so on. Just like Lisp implementations have done for 50+ years.
Another issue with Python is that the language semantics are much more subtle. This is an excellent talk that shows how even operations that appear simple in Python can be much more complex than they appear: https://youtu.be/qCGofLIzX6g
The other person he talks with is Andreas Gal and they talk about trace trees and method vs tracing jits.
What is "hidden class"? Is this a data structure?
The idea is that shapes (we represent objects like C like structs and not hash maps) are used for relatively non-dynamic objects and then we keep the shape metadata on functions themselves (inline caches) which means property access becomes just a byte offset.
Short example In JavaScript: Var a = {}; a.b = 15;
The hidden class object a maps the property b to where its stored, may include type information, etc.
Anyway, adding Neo was enough to differentiate the two names, so what's the problem? Let's not forget that Vim did the same thing to Vi when it was named "Vi IMproved."