Recommended path to read the Lua source code by the author of LuaJIT (2007) | Hacker News Reader

Recommended path to read the Lua source code by the author of LuaJIT (2007)

reddit.com

89 pointsjxub12 comments

Lua really is a fantastic example of how to design and write an interpreter. The style and details can be debated, but the high-level design is expertly crafted and something I wish more interpreters would take inspiration from.

I dislike the language, but the combination of its well-designed interpreter(s) and the general fat-free language design makes it the least shit of the popular scripting languages in my eyes.

luajit is also a very well designed JIT compiler, but I wouldn't recommend it for beginners because of how dense it is. Understanding luajit is comparable to understanding something out of the demoscene because of how much gets squeezed out of every line of code.

Do you have any suggestions for someone that doesn't know much about the subject but would be interested in learning more about it?

Long-term I'd like to be able to dive into stuff like V8 and actually understand its internals.

Mike pall wrote some posts in various mailing lists over the years that explain some of the things he did (but be aware that they only really scratch the surface)

https://news.ycombinator.com/item?id=11322060

https://news.ycombinator.com/item?id=2588696

http://lambda-the-ultimate.org/node/3851

The thing that makes luajit hard to understand is a combination of two things. First, the jit compiler uses advanced techniques that aren't easy to learn just looking at the code. You really need to know a lot of compiler theory to just get started. Additionally, mike pall wrote a large chunk of luajit in assembly language and hand-optimized lots of things, which gives the code that dense demoscene feeling that was mentioned before.

If you want to learn more about luajit one crucial thing would be learning about tracing jits for dynamic languages (luajit goes all in on tracing jit). The wikipedia article is pretty good. For papers make sure you read "Trace-based Just-in-Time Type Specialization for Dynamic Languages"

https://en.wikipedia.org/wiki/Tracing_just-in-time_compilati...

If you want to dive into a jit codebase one that might be interesting is the Higgs JIT. I haven't looked at the code itself very closely but I read Maxine's PhD thesis and her presentations on jit and I liked her approach of trying to make a jit that is as simple as possible. Most jits these days are massively complex beasts.

https://pointersgonewild.com/higgs/

Thanks for all the links, I'll be digging into em.

>Additionally, mike pall wrote a large chunk of luajit in assembly language and hand-optimized lots of things, which gives the code that dense demoscene feeling that was mentioned before.

The use of assembly is actually very tame in luajit. I was actually referring more to things like the fact that it spits out native code by iterating backwards so it can do register allocation in reverse (simultaneously and with minimal memory footprint!). Because of a lot of neat details of how the SSA IR is laid out and stored and how tracing JITs work, this results in more optimal register allocation in a way that is actually very straightforward to implement.

It's very, very clever and overall very clean with minimal code, but understanding why things are done in a particular way requires a lot of experience. The demoscene is similar: very neat 'hacks' that combine low-level programming knowledge with very deep domain knowledge motivated by the need to write code that is both fast and small.

Really hope we hear more from Mike Pall this year. He's been off the radar for way too long.

(2007)

His reading order is still good advice today though. Lua has seen many improvements recently but the file organization is unchanged.

(I can confirm this because I need to frequently dive into that codebase as part of my job)

Yeah, all o-kay; my comment was not meant as a dismissal but rather as a clarification. Because of various social and soap-opera-like-drama reasons in the community, I was super surprised and electrified to see such content assuming by default it was recent news (2018), given it was unmarked. Once I realized it was from 2007, as much as love the content itself, it became mostly not-news for me, and I cannot deny feeling to some extent misled by this omission.

Absolutely. I just wanted to point out the advice itself wasnt outdated.

Added, thanks!

Reply on news.ycombinator.com