LuaJIT 2.0 intellectual property disclosure (2009) | Hacker News Reader

LuaJIT 2.0 intellectual property disclosure (2009)

lua-users.org

103 pointsvortico26 comments

NaN tagging is really neat --- in LuaJIT, all values are stored as doubles; pointers are encoded as a NaN with the pointer part embedded in the NaN data field (you get about 50 bits to play with).

The beauty is that NaNs propagate; you can perform a complex numeric operation on them, and then test the result once while you're finished. If it's a NaN, then you know that one of the operands must have been a NaN and you escape back to the JIT for more analysis. If it's not... then it worked, and you just continue. And testing for a NaN is cheap.

NaN tagging exists since the dawn of time. CDC 6600 had reserved NaN mantissa bits... in 1963! The IEEE floating point standard just carried on these features. Many lisp implementation used NaN for packing data in the 80's/90's. Google for: lisp nan tagged data.

LuaJIT is great, and with the real fast and easy to use FFI it makes you design, prototype, test, and write real applications dealing with lots of "C" libraries much much easier.

I think the only real drawback, and maybe its being worked on (haven't checked the project in several months) is the internal memory limit of the lua heap (has to be below the first addressable 4GB's).

For those of us who are disbarred from looking at, or researching, IP: this is a statement by the owner placing this IP into the equivalent of the US public domain, or its equivalent in your state. The code is under MIT license. The author, obviously, doesn't indemnify you. If you're unfamiliar with the genius of the FFI in Lua, or Mike Pall's JIT/VM wizardry, it is well worth looking at.

Is Lua easier to optimize than Javascript? Is impressive how fast LuaJIT is for being a one man show.

Yes. While they have very similar semantics, Lua is just simpler. (ES6 though goes way beyond and is probably quite a bit harder to optimize simply because of having much more surface area.)

Of course, most of LuaJIT is applicable to JS as well. Mike Pall simply implemented it better than whole teams at Google and Mozilla. Yeah. Dude's pretty smart.

I think typical ES6 code is actually less likely to use parts of JavaScript that are difficult to optimize because we now have a lot of features that let us be more clear about our intentions. Proxy objects are probably more difficult to deal with, but the rest of ES6 is probably easy to take advantage of.

> ES6 though goes way beyond and is probably quite a bit harder to optimize simply because of having much more surface area.

I don't know, the ES6 inclusions aren't generally increasing the data model complexity.

As smart as Mike Pall is, it isn't just that he's some crazy genius. Lua lacks a mutable prototype chain which can really spoil javascript performance and optimizability.

As far as I can tell you can't something like:

Array.prototype[0] = 5;

and make [] the same as [5].

There's a lot of edge cases in javascript that make its optimization much more complicated than lua's.

You can create a prototype chain in Lua by associating an object with a metatable that has an __index metamethod. If the table that __index references has a metatable itself, the chain continues.

http://www.lua.org/pil/13.4.1.html

Importantly, `tbl[key]` only consults `__index` if `key` isn't actually a key in `tbl`, so keys in derived objects override keys up the prototype chain.

Just in case, anyone has somehow gotten to the conclusion that Lua's semantics are 'simple', they should closely inspect this example and try to figure out through which contortions the VM has to go to make this work:

    local t = setmetatable({}, {
      __index = pcall, __newindex = rawset,
      __call = function(t, i) t[i] = 42 end,
    })
    for i=1,100 do assert(t[i] == true and rawget(t, i) == 42) end

[LuaJIT has no problems with this code and turns it into 8 machine code instructions for the actual loop.]

Anyway ...

This permanent excuse of JavaScript proponents that it has more complex semantics, which somehow prevents it from being made fast, is getting old. There are no insurmountable obstacles to make JavaScript fast -- it just takes more effort!

And they dug this hole themselves, by not cleaning up the language and allowing new complicated features into the language. Well ...

Also in Lua you can change the prototype of constructed objects, by just calling setmetatable() on them again; while in Javascript you can't --- once the object's been constructed, that's it.

http://stackoverflow.com/questions/7015693/how-to-set-the-pr...

I ran into this a little while back while trying to port some Lua code to Javascript. It was very annoying.

> while in Javascript you can't --- once the object's been constructed, that's it.

ES2015 standardised the read/write __proto__ attribute (which most browsers had implemented for a long time), and added a #setPrototypeOf call. The first answer of your link notes exactly that.

It's added as an optional feature, as the first sentence of that answer describes:

> ...the answer below is no longer accurate. _proto_ is being added to ecmascript6 as "normative optional" which means it isn't required to be implemented...

It can't always be polyfilled, either, which makes the feature effectively useless. I'm not even sure why they added it as such a half-hearted feature to ES6; I suspect politics (Brendon Eich appears to have been dead set against having mutable protos for years).

> It's added as an optional feature

Fuck's sake mate, Object.setPrototypeOf is not optional (and __proto__ is only optional for non-browser environments).

> It can't always be polyfilled, either, which makes the feature effectively useless.

That's breathtaking dishonesty.

  Array.prototype[0] = 5;
  and make [] the same as [5].

Huh? I don't know what you mean by this.

In lua you can easily replicate javascript's prototype inheritance.

Being able to build it on top of the language and having it right inside basic language elements as an always-on possibility are quite different things.

The former allows straight forward optimization of the simple case, simply accepting the performance hit of the stuff built on top (which still might benefit from some of those optimizations), while the latter forces the optimizer to invest a large effort into distinguishing between the simple case and the exceptional, and then there still needs to be an always-on fallback path from simple to exotic.

"Prototype inheritance" is a fundamental building block in Lua, enabled via the metatable mechanism. It's by far the most common way to do OOP in Lua. I'd say it is, in fact, built right in to the basic language.

One big factor is that LuaJIT has a really fast non-jitting interpreter. There is some overhead on JIT compilation so its best to avoid it unless you really are in a hot loop.

On the other hand, in Javascript the language semantics is branchier and overall more complicated so its hard to write a fast base interpreter. The system relies much more on the JIT compilation, which also means that Javascript JITs have evolved to be much more complex than LuaJIT's JIT. LuaJIT bets all its chips on pure tracing compilation while Javascript JITs tend to go for more hybrid strategies nowadays.

This is an interesting summary of the technology used in LuaJIT.

Even more valuable when you consider that Pall has stepped down as head of LuaJIT dev and handed off maintainership to others.

He's still head. He hasn't found a suitable replacement yet.

Oh interesting. I heard that CloudFlare was going to take over, but that may just have been about "we will handle paying the salary of the right person once they are found".

The current situation is that 2.1 (currently in beta) is finished with Mike on the board, and after that Mike is "out".

Is that even possible? :)

As somebody writting a VM that is simular to LuaJit as a hobby project, I have read this document over and over again. There are so many interesting ideas and approches.

Reply on news.ycombinator.com