Moonjit – Fork of LuaJIT to Continue Development | Hacker News Reader

Moonjit – Fork of LuaJIT to Continue Development

github.com

119 pointsxvilka58 comments

I really hope this gains traction. i use Lua in a professional setting on an smaller embedded Linux system. Lua's memory requirements, ability to bind with C, and developer speed really works well for this project.

But it feels like a strain on the Lua community to have LuaJIT essentially "stuck" with Lua 5.1. Many projects seem to have taken a long time to move on and start moving forward to 5.2, or 5.3 which were released in 2011 and 2015 respectively. LuaJIT is so impressive it feels unfortunate to leave that community behind and move forward.

When we evaluated Lua, and which Lua version to go forward to this was our key points:

1. LuaJIT was abour twice as fast for us. 2. LuaJIT has nice C integration/binding. 3. LuaJIT was no longer maintained/developed/etc and was now ~10 years out of date wrt to the core language. 4. Lua is being used in non-performant areas, we have C when we need performance. 5. Integers and bitwise from 5.3 operations are important for us. 6. Overall the changes in 5.2 and 5.3 are good language improvements.

We ended up going with Lua 5.3 due to ongoing development, and for us it ticked all the boxes. But it would amazing to have the options to use LuaJIT, and to encourage everyone to move forward instead of being stuck on 5.1.

LuaJIT includes bitop, which has similar semantics to javascript's built-in bitwise operators. If you want to avoid writing `bor()` expressions it seems like a preprocessor that targets luajit is a better answer than giving up the LuaJIT FFI.

Some comments: 5.1 with LuaJIT extras is arguably the superior language version so the vast majority of Lua devs do not see it as a downgrade.

In fact I dare say that it is 5.3 and subsequent versions that are and will remain the oddities. LuaJIT is just too good.

Lua 5.3 is technically a language fork as basic functions now have new behaviour. So you would still have to decide even if LuaJIT did not exist. The thing with LuaJIT is a symptom, not a cause.

There are so many forks of LuaJIT going on (raptorjit, luavela, moonjit, openresty's luajit 2, shine, ... The bad thing is that each one tries to fix a different issue: compatibility with new versions of the language, eliminate the 2 Gb limit, make more maintainable code, add extensions for web handling,...

And then, some of them introduce new problems such as limiting the number of available platforms.

FWIW, if RaptorJIT had not ripped out arm64, I would have ended up recommending replacing Luajit with RaptorJIT in Fedora and not bothered with forking moonjit.

Indeed, I've been looking at writing a new backend (ambitious and foolish, many have tried, I know), so I was looking for the hottest fork, but RaptorJIT specifically guts most of the backends, and the other forks were not really doing anything.

RaptorJIT has a mostly finished port of the bytecode interpreter to C. That should make new arches /much/ easier to support because you don't have to hand-code a VM in asm, only write a JIT assembler backend. Check it out at https://github.com/raptorjit/raptorjit/pull/254

Current status is that it passed the LuaJIT test suite but that's not 100% coverage and it doesn't call into the JIT yet.

What sort of performance have you been getting?

Why not use the openresty fork which looks to be properly maintained?

That was my second fallback until a couple of months ago and in fact I did some work for openresty too for luajit. Openresty luajit2 however has a bunch of extensions that are incompatible with Lua and luajit, making it unsuitable as the stock luajit replacement. It would be possible to make those extensions conditional on build flags but the blocker there was the fact that the maintainers did not seem keen on doing releases and more importantly, moving too far away from luajit. Overall it was just simpler to start doing releases myself to fulfill a need (a more central place to maintain fixes for distributions) and then work towards a convergence.

Out of interest, which one removes the 2 GB limit? That was a reason I used the standard lua a while ago.

There's a patch for LuaJIT that increases the limit. It comes pre-applied with the LuaJIT that installs with openresty. https://github.com/openresty/lua-nginx-module/issues/1019

There is no 2GB limit if LuaJIT built in GC64 mode. It was an opt-in build time option until 13 days ago now its on default for 64 bit builds https://github.com/LuaJIT/LuaJIT/commit/bd00094c3b50e193fb32...

Still amazes me that Mike Pall was a pseudonym. I'd really love to understand what they do for a day job.

Luajit is a brilliant project.

“Mike Pall was a pseudonym”

Any source on this??

Some pointers: https://news.ycombinator.com/item?id=18765514

It says “not any Lua stuff”, though. Presumably that would mean “Mike Pall” is their real name?

Blows my mind. A huge lesson in humility

if you google "AFLG C64" you'll see historical references to the author, one "M. Pall". I think your link indicates the exact opposite of what you're suggesting - that "Mike Pall" is his real name and that it was the "other stuff" that was published under a pseudonym.

IDK why this myth of him being a pen name persists.

As you pointed out, he publicly denied those claims.

He also put his postal address onto the luajit homepage. And it adds up with his history at the University of Karlsruhe. You can find pointers that he was actually there in 1994, from old websites that survived the decades. His Karlsruhe e-mail address is even still being credited in linux's net/ipv4/tcp.c. The best proof I found that he's an individual is him being credited as supervisor of an academic Dipl. Inf. thesis that's still available on the wayback machine: https://web.archive.org/web/20000919224408/http://andy.rz.un...

I doubt that joking goes so far that you are mentioning a pen name on a thesis front page. In addition to the front page containing his name, the thesis also contains screenshots of websites the wayback machine hasn't caught in time that contain his name.

That post (mine) was either posted in a very confused state or the opening line is simply misstated, since, as others have noted, even the quoted text implies the opposite!

Amazing!. Didn't knew that. Its like Satoshi Nakamoto. Another anonymous genius.

Mike Pall is 100% real. No pseudonym. No group. He's an old school German hacker that previously contributed to Linux and a bunch of other open source software in the 90s.

So much about people not believing in 10x engineers. They are very much real, along with Fabrice Bellard.

Btw, LuaJIT is certainly not dead. Mike Pall just recently added lots of commits. https://repo.or.cz/w/luajit-2.0.git

This has to be upvoted more! There was already a lot of talk about LuaJit being unmaintainted last year. When Mike Pall was even more active.

Right. People prefer to post bullshit instead (e.g. that Pall was no real person, Lua 5.3 being superior, and the like).

If Mike abdicated his post, I would prefer to take him at his word.

What do you mean by that? Don't understand.

He publicly posted that he was stepping down from maintaining luajit and did not subsequently post anything saying that he had changed his mind.

Well, he doesn't owe anybody anything. On the contrary, I think it's generous that he still cares about the project.

Sure. It's appropriate to say that the project is unmaintained if he doesn't plan to maintain it.

It's not unmaintained. There are at least three companies doing their own maintenance and development on a professional and organized level. Most even open source their results. And there are also individuals and groups working on forks. There is no "central institution" organizing a "main repository" as in Linux, but there is relevant progress and the will for a stable product in each of the forks.

I worked at Cloudflare, sometimes on luajit. When I find myself merging patches from random lua community members that will never be in an official point release to fix crashes in well-behaved lua code, I'm not sure what other word property describes the situation.

It may warm my heart to know that lukego, yichun, and siddhesh are also merging the same patch, but it doesn't really help.

Mike is a real person from Munich, but his name is certainly not Pall. That's what they mean. The rest is bullshit: 5.3 being superior, and such.

What makes you believe his name is not Pall? I have other information.

I remember a project with a similar philosophy:` https://github.com/raptorjit/raptorjit

RaptorJIT seems doesn't care about compatibility.

LuaJIT uses DynASM, which is also stuck (e.g. no AVX-512). It would be great to see DynASM development continue.

This is not needed, almost nobody uses that, not even in intrinsics. Due to CPU downclocking. Only the recent AMD/Arm/Power SHA extensions might be useful to be added.

i’ve always read about how the lua jit was a technolical marvel ( without understanding why but that’s another topic). If that’s the case, then how come it isnt’t maintained anymore ?

Uh? Being a 'technical marvel' has little relationship with making money and without money developers may choose to do something else whenever they want..

IIRC there's a comment from Mike Pall (maybe it was here, can't bother to search it for now) where he stated very clearly that there was always enough money on the luajit table. So no financial reasons, he just wanted to do something else (and he didn't like the new lua features so much, pain to implement and all that stuff, so maybe that's the real reason...).

Features in PUC-Rio Lua after Lua 5.2 or so are mostly all terrible, with the possible exception of <toclose> (a bad version of golang's defer). So I respect the choice not to add those features. There was still lots of work to do to improve the GC and such though.

What about integers and bitops in 5.3? That seems like a big deal

Bitops are in LuaJit

Lua 5.3 integers are 64 bits long by default, and that makes a real difference.

That's trivial to implement as a C function though, so no real need to add it as a language feature

At the time I'm writing this, the latest commit in https://github.com/LuaJIT/LuaJIT was 26 minutes ago. There has also been a constant stream of commits over the last months (I didn't check further), but no frantic activity. Almost all commits in the moonjit history also seem to be be cherry-picks of these LuaJIT commits.

So calling LuaJIT unmaintained, and saying that moonjit "continues development" both seem a bit exaggerated to me.

The last release was a beta from May 2017.

The last time Mike Paul pushed anything is almost a year ago until two weeks ago based on the two RSS feeds I use. The Git commit dates could be unreliable from either cherry-pick shenanigans or him just not bothering to push the changes until two weeks ago.

Maybe compare your feeds with:

https://github.com/LuaJIT/LuaJIT/commits/v2.1

I see quite regular commits there?

The v2.1 branch of moonjit specifically aims to remain "luajit-like" in the sense that it is the explicit goal to make sure we don't add incompatible features and we definitely don't make it so that if Mike Pall decides to continue development after a year (which he appears to have done, although one can never be sure given his record over the past few years), we can easily enough merge patches.

The master branch is where moonjit development is more active with people adding 5.2 and 5.3 features and new architectures too. It is in a way a different direction from luajit (since Mike specifically stated that he intends to remain incompatible with stock Lua because his dialect is better) and hence may not necessary converge with luajit if it ever actually comes back to life.

Calling LuaJIT unmaintained isn't exactly true. It's still getting commits, however the author has stopped trying to keep it up to date with the mainline Lua updates. Lua has a history of not caring too much about backwards comparability, so the LuaJIT devs have decided to stay with Lua 5.1, even though 5.1 is nearly 10 years old, and 5.4 is on the way soon.

The tragedy is that no game console vendor I'm aware of will let you use JIT codegen due to security concerns.

Tried to find out something about your roadmap. Would be helpful if you could add additional information somewhere. Is this a continuation of LuaJIT 2.1 or 2.0? What short and longterm goals do you have? Why did you choose a new name for it?

Really glad to see someone taking up the mantle! My dream Lua would support multicore parallelism and arbitrary precision arithmetic out of the box.

Reply on news.ycombinator.com