LuaJIT 2.0.5 and 2.1.0-beta3 released | Hacker News Reader

LuaJIT 2.0.5 and 2.1.0-beta3 released

luajit.org

145 pointsmarksamman67 comments

LuaJIT powers OpenResty[1] which powers my web framework Lapis[2] which powers my company itch.io[3]! (wow that's a mouthful)

Anyway, it's a stack I'm really happy to be using. Asynchronous IO without any code nesting. I wrote more about the benefits here: http://leafo.net/posts/itchio-and-coroutines.html

[1] https://openresty.org/en/

[2] http://leafo.net/lapis/

[3] https://itch.io/

I also made web server infrastructure for the Lua package manager LuaRocks in the same stack: https://luarocks.org Source is on github: https://github.com/luarocks/luarocks-site

Lua (especially with LuaJIT) is a great choice for web development.

Love Lapis.

Question, since Lapis just transcode Moonscript into Lua to be used with OpenResty - why is Lapis so much slower than raw OpenResty/Lua?

https://www.techempower.com/benchmarks/

Because the lua code moonsript emits is inherently slower.

>> "inherently slower"

What's "inherent" in moonscript that makes it slower?

Not necessarily. Like CoffeeScript, it depends on what features you use.

Not really, the code mapping is very straightforward and generally has no performance degradation. There are some language constructs that are implemented in a way that can create anonymous functions in order to convert statements into expressions. Future versions of MoonScript can do a better job to eliminate those. If you're aware when they're generated you can avoid them by writing as you might do in Lua. Realistically though, the performance difference is negligible in comparison to all the other things that might happen in your average web request.

"slower" : I see Lapis is ahead of most frameworks, except platform or micro frameworks coded in C++, C or java. I think people should notice instead it's one of the fastest frameworks?

Did you compare Lapis to OpenResty in the benchmark above?

In case you're not aware Lapis = Moonscript + OpenResty.

And Moonsript is like Coffeescript but for Lua

Lapis ~= Moonscript + OpenResty, Lapis = OpenResty + loads of framework stuff to make development easier, as Leafo explained.

You can use Moonscript, but you can also use pure Lua.

Lapis is an entire web framework, OpenResty is a relatively low level interface to nginx. Lapis is an abstraction on top of OpenResty's Lua interface to make developing large web applications much more manageable.

It has nothing to do with MoonScript. MoonScript generates speedy code.

Any plans to focus on increasing Lapis performance to be closer to raw OpenResty?

Thanks for all your work Leaf. I use and recommend Lapis.

you also made moonscript and mustache.php. I imagine next you might rewrite nintendo from scratch in lua, and singlehandedly control the entire games industry.

Is there something like uwsgi for Lua?

http://uwsgi-docs.readthedocs.io/en/latest/Lua.html

LuaJIT is such an amazing piece of software - apparently to such an extent that only Mike Pall fully understands some of its internals. Besides being much faster than the official Lua VM, it offers extended features like the FFI library [1].

Kong[2] uses it extensively, by leveraging the underlying OpenResty[3] framework.

[1] http://luajit.org/ext_ffi.html

[2] https://github.com/Mashape/kong

[3] https://github.com/openresty/lua-nginx-module

> LuaJIT is such an amazing piece of software - apparently to such an extent that only Mike Pall fully understands some of its internals

Fully agree about the awesomeness of LuaJIT and incredible capabilities of Mike Pall, but also the reason I've stopped considering using LuaJIT for projects that should be "future proof". I think that depending on technology which can only be maintained by one genius is too much of a risk.

Looking at the LuaJIT changelog [1], it's interesting how many bugs need fixing in each version: 20 in 2.0.5 and 32 in 2.0.4.

Compare this with PUC Lua [2], which seems to have very few bugs: only 4 in 5.3.3 and 3 in 5.3.2.

LuaJIT and PUC Lua appear to be very similar superficially ("LuaJIT is Lua, but fast"). The relative bug counts reinforce the fact that in the details, the two implementations are very different - for example, LuaJIT uses a lot of assembly language while PUC Lua is pure C89.

The advantages of LuaJIT over PUC Lua seem to be performance and its FFI. I thought PUC Lua's only advantage was portability, but perhaps PUC Lua is also a generally more "correct" implementation?

[1] http://luajit.org/changes.html

[2] https://www.lua.org/bugs.html

Not only does LuaJIT have platform-specific assembly in its source, but a big part of its advantage over PUC Lua is of course that it's compiling down to machine language on the fly. And that process is also platform specific. I have definitely seen bug fixes along the lines of "fix X for Y platform's JIT output."

I have also seen non-jit-related bugs impact us in production.

BUT, LuaJIT really is FASTER. Even when JIT cannot be used (e.g. on iOS), it can still be several times faster than PUC Lua.

And that speed is a real killer feature for us. Otherwise, we'd just use JavaScript.

Do you find LuaJIT much faster than a JavaScript engine like V8? That used to be true, but I was under the impression that V8 had basically caught up (unless you need to make a lot of FFI calls, since LuaJIT is heavily optimized for that).

I may be behind the times, but last I checked there was no way to use JITed JavaScript in iOS apps. Or maybe you could put JS in a hidden webview, but bridging to native was extra slow with this approach?

While LuaJIT can't be JITed either on iOS, I think the general consensus is that non-jit Lua is faster than non-jit JS.

I do a lot of bridging Lua to native, and that's another area where Lua is often considered more efficient.

Again, these thoughts may be out of date.

Aside from being incredibly low on bugs (often, there are no known bugs at all, although complaints are likely to be treated as feature requests rather than bugs), PUC Lua is also far smaller, and vastly easy to hack on. There are various patches available for it. On the other hand, the guts of LuaJIT are difficult to say the least. Most opportunity to eek out a little more speed by making the code twice has dense has been taken. However I found that adding custom library routines to LuaJIT (for operations like concatenating tables) was actually pretty easy as long as you don't want to be able to JIT them (since library routines call into an internal API which is well shielded, and there's still a lot of PUC Lua code in LuaJIT).

Also, LuaJIT has other extensions aside from FFI; see the website.

If you want to get an idea just how good LuaJIT is take a look at these benchmarks against the stock VM(which is already good) [1][2]

We used to run Lua on a 300MHz MIPS processor and it was impressive in how well it worked on that constrained platform(only 8mb is system ram, of that Lua ran in a fixed 400kb block).

[1] - http://luajit.org/performance_x86.html

[2] - http://luajit.org/performance.html

The most impressive part of LuaJIT is that its non-JIT'd interpreter is several times faster than Lua in benchmarks. Great for platforms like iOS which don't allow code modification at runtime.

Didn't Mike Pall step down sometime last year? It looks like he's made a ton of commits [0] and also made the announcements on the mailing list [1]. Anyways, congrats on the new release!

[0] https://github.com/LuaJIT/LuaJIT/commits/master

[1] https://www.freelists.org/archive/luajit/05-2017

The commit author doesn't tell the whole story: the long description on most of the recent commits mention somebody else.

For an editor built from the ground up using LuaJIT see https://howl.io

[/plug]

The docs are pretty. What do you use to generate e.g. https://howl.io/doc/spec/interact_spec.html ?

Thank you. I believe the tests are converted to HTML by a custom Ruby script: https://github.com/howl-editor/howl/blob/master/site/bin/spe...

The manual and API docs are hand written in markdown.

If we're plugging things that can benefit from LuaJIT: I wrote a console-based mail-client, kinda like mutt, which uses Lua for all the scripting and UI.

https://lumail.org/

New release will be out in a few days.

The documentation could use a section explaining what Howl has to offer over other text editors than are extensible using Lua, such as Textadept (https://foicica.com/textadept/).

Good point. It's hard to highlight certain differences because it's a very different editor. In general I think Howl offers a better API and a more keyboard centric UI compared to Textadept. Most commands are operated via the command line which offers auto-complete (vs operating via the menu bar, with custom dialog boxes for each command). Howl also doesn't use Scintilla (Textadept does) and is not subject to Scintilla's limits.

Really can't understant why Lua isn't more used. It play the same role as Ruby, Python and JS and is very fast, at least, much more then Ruby and Python.

Batteries are not included in Lua. To do most anything interesting requires third party libraries (like directory manipulations, network, etc).

Yep. Lua's small (and removable) standard library makes it really great for embedding, not so great standalone.

But when you need batteries, you just install luarocks and move on with it.

It's not that libraries are not available, it's that they are not _standard_. Python has much healthier and more coherent community partially because everybody started out with same carefully chosen building blocks. With Lua you either build your own or bring in dependencies from libs and frameworks.

Regardless, I'm enjoying Lua more and more and it's becoming my go-to language for prototyping and exploring concepts. The code just flows more naturally than in other languages.

I think there is something else to this. Most highly productive Lua applications I've seen, been involved in, &etc., are bound to system-level libs/api's, etc. So the thing about Lua as a scripting language like Python, is that you can use it like that. But the major power of the language is when you apply it by glom'ing it into some bundle of C/C++ libraries that are, indeed, BYO-batteries...

Like, Lua is bring-your-own battery, true. For everything else, there's Penlight/luarocks, though.

and how good is the ecosystem? honest question

It's much smaller community than Ruby or Rust... People are very friendly and most of learning content is freely accessible. Most of us use LuaJIT (at least on desktops) but many are worried about its future, as lead genius has stepped off and few other people can understand the internals of LuaJIT.

There's a very nice "package manager" for downloading and installing modules, called LuaRocks. On the other hand, lack of tooling is a big issue. ZeroBrane seems to be only decent IDE with debugging support.

There's ton of stuff in libraries and Lua extensions, you just have to be careful about what you are integrating. You might for example end up with two incompatible implementations of class system from two different libs.

Isn't it the same in many other languages? For a Ruby project, you usually have to use a dozen gems to get started. For a javascript project, it's at least two dozens of npm packages

But at the very least, you can open a socket without external modules in those languages.

It's comparable to mruby but better since you can install extensions w/o rebuilding your interpreter.

Not just that, but Lua is very conservative about adding new language features so a lot of quality of life things are missing - no ternary operator, verbose lambda syntax, the incoherent mixing of arrays and dictionaries, bad default scoping, stuff like that.

If you are willing to fork Lua (and it's been done---one example is Ravi, which brings static typing) you can fix the verbose lambda syntax (http://lua-users.org/lists/lua-l/2017-04/msg00387.html) and add ternary operators (which I haven't missed). I haven't found the mixing of arrays and dictionaries that bad in practice (and I do a lot of Lua programming at work) and the default scoping ... yeah, I can see that (I've learned to deal).

I had a fork a long time ago solely to allow the use of "?" in function-names. Writing:

    function alive?
       ... blah
    end

Is so much more natural, after being exposed to ruby. But maintaining forks is hard, so I've slowly gotten use to the standard.

It's a shame that the development is so conservative, and closed, but despite that it is a great language to play with.

    A function that tests for something involving its arguments
    is called a predicate and usually ends in p or -p.

http://www.cliki.net/Naming+conventions

>"incoherent mixing of arrays and dictionaries"

Incoherent? 10 years of Lua development, never had a problem with it. In fact, the ability to use a table as either an array or a dictionary is one of the great features of the language. Most newbie Lua coders get over the differences pretty fast. For everyone else, there is lua-enumerable.lua ...

https://github.com/mikelovesrobots/lua-enumerable

Well, 10 years isn't the problem, it's picking it up in the first place. For example, the whole pairs/ipairs thing falls out of that original weird decision.

I love Lua. It's embeddable, it's written with newbie-friendliness in mind... but I still think it has flaws that I wish we'd see fixed. To me, the biggest one missing is just some simple convenience stuff for higher-order-functions - like I said before, the lack of a ternary "inline if" which is super-handy for writing one-line lambdas, and the verbosity of its lambda syntax.

I dunno, I think that the distinction between pairs/ipairs is quite handy - there are times when you want to use an array, a list, a dictionary, a sparse array, a sorted array, and so on. Lua's tables are so flexible that it can be used in all of these cases quite handily - of course, the programmer has to pay attention to how he's using those tables, though. I think thats kind of a strength, but ymmmv ..

I've written a few programs in both Lua and Python, and the Lua is generally almost twice as long. Lua tends more forgiving of mistakes which I'd rather be errors, like a missing table key. So though it's great in some ways, it's not my first choice to get something done with.

Build system in pure Lua: https://github.com/rweichler/aite

3DS port: https://github.com/rweichler/luajit-3DS

iOS runtime inspector: https://github.com/rweichler/lucy

Cydia alternative: https://github.com/rweichler/jjjj

And this is just stuff by me, some schlep. LuaJIT is so underrated it's kind of sad.

Don't forget the Not Yet Implemented features: http://wiki.luajit.org/NYI

Edit: By the way, can someone explain why ipairs() is implemented, but not pairs()?

Those are just features that aren't JIT compiled. They are still implemented, in the sense that LuaJIT fully supports them in the interpreter, so you can use them in LuaJIT programs.

I'm guessing that pairs() is not JIT-compiled because iterating over a hash table involves logic that is significantly more complicated than iterating over an array (like ipairs()).

>By the way, can someone explain why ipairs() is implemented, but not pairs()?

https://www.freelists.org/post/luajit/LuaJIT-21-status-and-s...

I'm still unable to use LuaJIT because the build system makes an incorrect and silly assumption that code compiled during the build by the build compiler had the same executable properties as the code compiled during the build by the host compiler, leading to the situation where: repeatable builds don't work, and you can't compile for Linux/x86_64 from Linux/x86 or Linux/arm.

It is one of very few packages broken this way, along with basically only XFree86 4.8.0

:-(

I hooked up LuaJIT to emacs for fun. Emacs recently released a new module system where you can build DLLs that are loaded dynamically: http://diobla.info/blog-archive/modules-tut.html

Getting LuaJIT to work on 64-bit OS X required building a custom emacs with linker flags -pagezero_size 10000 -image_base 100000000. This is unfortunate since obviously users will not take the time to build an emacs with custom linker flags. But more generally, this is an issue any time you want to embed LuaJIT in an application plugin rather than the application itself.

There's a bunch of scattered info about the issue:

https://gist.github.com/nddrylliog/8722197

http://hacksoflife.blogspot.fr/2012/12/integrating-luajit-wi...

https://en.blog.nic.cz/2015/08/12/embedding-luajit-in-30-min...

http://luajit.org/install.html

I was hoping there might be a way of sidestepping the issue somehow. If anyone has any ideas, it would be appreciated. LuaJIT has some nice speed improvements compared to standard Lua, but more importantly it has a pretty amazing FFI library. The C declaration parser in particular is valuable.

(Yes, I know it's pointless to hook LuaJIT to emacs. It's mostly a "just because" project.)

The solution is included in 2.1.0-beta3: "LuaJIT for x64 can optionally be built for LJ_GC64 mode by enabling the -DLUAJIT_ENABLE_GC64 line in src/Makefile or via 'msvcbuild.bat gc64'." "This mode removes the 32 bit limitation for garbage collected memory on 64 bit systems." [0]

[0] https://www.freelists.org/post/luajit/LuaJIT210beta3

Perfect! Thank you.

> I was hoping there might be a way of sidestepping the issue somehow. If anyone has any ideas, it would be appreciated. LuaJIT has some nice speed improvements compared to standard Lua, but more importantly it has a pretty amazing FFI library. The C declaration parser in particular is valuable.

Why use LuaJIT over plain Lua here? I can't imagine that you need the extra speed in a text editor. The ffi is available for normal lua too: https://github.com/facebook/luaffifb/

Because it's so easy. From the manual:

So here's something to pop up a message box on Windows:

    local ffi = require("ffi")

    ffi.cdef[[

        int MessageBoxA(void *w, const char *txt, const char *cap, int type);

    ]]

    ffi.C.MessageBoxA(nil, "Hello world!", "Test", 0)

Bing! Again, that was far too easy, no?

Compare this with the effort required to bind that function using the classic Lua/C API: create an extra C file, add a C function that retrieves and checks the argument types passed from Lua and calls the actual C function, add a list of module functions and their names, add a luaopen_* function and register all module functions, compile and link it into a shared library (DLL), move it to the proper path, add Lua code that loads the module aaaand ... finally call the binding function. Phew!

As I linked in the above post, the ffi is available for normal lua too: it's not exclusive to LuaJIT

In fairness, luaffifb seems to give the same feature.

I think the main argument against Lua + luaffifb is that it might require an external dependency: luarocks. Whereas if someone were to theoretically provide Lua integration with emacs, it might be a good idea to offer the ffi library as a baseline feature.

This is a weak argument, because it should be possible to embed luaffifb plus Lua inside of a linked library, with no need to depend on luarocks.

It's probably best to explore both options.

Well, LuaJIT will also JIT compile Lua code operating on C structures - so speed could be another big argument.

I did write a Lua binding to TCC [1] and used that to write a Lua module to load Lua modules written in C directly from source code [2], which I used in an older version of my JSON decoder [3], which contains the C code embedded right in the Lua source code.

On the down side, TCC doesn't support all the architectures I use, so I mostly use it for proof-of-concept and throw away code.

One thing I keep thinking is to possibly use TCC (or parts of it) to parse C header files directly, so it would be easier to use an FFI in Lua.

[1] https://github.com/spc476/lua-conmanorg/blob/master/src/tcc....

[2] https://github.com/spc476/lua-conmanorg/blob/master/lua/cc.l...

[3] https://github.com/spc476/LPeg-Parsers/blob/json-1.0.0/json....

> LuaJIT has some nice speed improvements compared to standard Lua, but more importantly it has a pretty amazing FFI library. The C declaration parser in particular is valuable.

My mouth was hanging open when I discovered that you could literally a verbatim header with packed structs and use it and it would work. Then I looked at the source to find out what they were using as a compiler frontend; it's all hand written. Incredible.

As an emacs user, I would love if its lisp interpreter were sped up or replaced with something faster. There's a lot of computationally expensive stuff written in elisp. Even for text editing, it can be laggy when parsing large files of source code for IDE purposes.

Reply on news.ycombinator.com