LuaJIT uses rolling releases | Hacker News Reader

LuaJIT uses rolling releases

freelists.org

93 pointsP_qRs55 comments

Also interesting is that https://luajit.org/status.html now states “LuaJIT is actively developed and maintained” (whereas for the last ~5 years, “actively” isn’t a word I’d have used), and makes reference to a TBA development branch.

This makes me soooo happy to hear because with all of the forks of LuaJIT post it not being maintained by Mike ~5 years ago, no one seems to carry the baton very well.

LuaJIT is truly an engineering marvel that more folks should adopt.

If it's actively being maintained again, hopefully that will happen.

What I don't quite get is the follow:

"Please note: The main LuaJIT author (Mike Pall) is working on unrelated projects and cannot accept bigger sponsorships at this time. But other community members may be open to sponsorship offers — please ask on the LuaJIT mailing list for any takers."

https://luajit.org/sponsors.html

"If you want to pay someone to make a change to LuaJIT, Mike Pall is not available, but other people on the LuaJIT mailing list may be."

https://repo.or.cz/luajit-2.0.git/shortlog

That's some good news I needed but didn't deserve!

The RISC-V port is marked as TBA, too.

> DO NOT use pseudo-releases or tarballs created by third parties. In particular, automatic tarballs made by GitHub and other mirrors are lacking the .git directory and will not properly build anymore.

It's a little concerning that their build process relies on a .git directory being present. That's going to make it quite tough for packagers to actually package this.

I think https://github.com/LuaJIT/LuaJIT/commit/6a2163a6b45d6d251599... improved things a bit, notably making automatic tarballs work again.

The export-subst attribute was new to me.

Here is a description of what it does:

If the attribute export-subst is set for a file then Git will expand several placeholders when adding this file to an archive. The expansion depends on the availability of a commit ID, i.e., if git-archive[1] has been given a tree instead of a commit or a tag then no replacement will be done. The placeholders are the same as those for the option --pretty=format: of git-log[1], except that they need to be wrapped like this: $Format:PLACEHOLDERS$ in the file. E.g. the string $Format:%H$ will be replaced by the commit hash. However, only one %(describe) placeholder is expanded per archive to avoid denial-of-service attacks.

https://git-scm.com/docs/gitattributes#_export_subst

Surely a packager would be working off of the actual releases not a random git archive?

And if they can’t wait for official releases, they’d be working off of the git repo?

> Surely a packager would be working off of the actual releases not a random git archive?

There are no official releases. To quote from the linked email:

> No release tarballs or binaries are made available. Please do not use obsolete versions from older tarballs or zip files. Please remove any outdated links to these downloads -- they will cease to work soon.

The only way to get a release version of LuaJIT is to compile it from the git repo.

> The only way to get a release version of LuaJIT is to compile it from the git repo.

It will be interesting to see how this is packaged in, say, Ubuntu. Will they assign their own version numbers? Dates?

I would say something along the lines of git-describe, and packages would need to depend on the exact version.

If upstream can't commit to compatibility across release versions then I don't see it happening in distros either.

Sounds like eternal beta.

It has been an eternal, abandoned beta for many years. But sounds like work resumed now. Maybe one day it shall be stable.

The packagers can make their own snapshot, but they don't want to grab a full .git directory for the source - distributions such as Debian never want to do that: The packager is responsible for auditing all the code including copyright status in the artifacts - best not have the whole history of the project included in that case, because the task is then a little bit too big.

You can clone a specific tag or branch from a git repository without the full history. It looks like LuaJIT's git repository does not support this, although the github mirror does:

# git clone -b v2.1.ROLLING --depth 1 git@github.com:LuaJIT/LuaJIT.git

The git log looks like this:

    commit 2090842410e0ba6f81fad310a77bf5432488249a (grafted, HEAD, tag: v2.1.ROLLING)
    Author: Mike Pall <mike>
    Date:   Mon Aug 21 03:18:35 2023 +0200
    
        Merge branch 'master' into v2.1

Somewhat difficult to find on the linked list archive, but this exact issue was dicussed there. See the followup mails: https://www.freelists.org/archive/luajit/08-2023?threads=1

I'm confused what you mean. What is a packager doing here? I always assumed that the intention of this library is that you, as a developer, compile and embed it in a product that needs a scripting engine. What use would a standalone LuaJIT package be?

It's available as a package, and a library, in most Linux distributions. It includes the library and the interpreter.

edit: I have a few scripts with #!/usr/bin/luajit as the shebang line.

Interesting, that's neat. Somehow didn't realize using it like this was an option, but now that you mention it...

libluajit is a drop-in replacement for the regular liblua (v5.1)

Even ignoring the installed interpreter, libraries on Linux distributions are usually installed system-wide as their own packages which other packages depend on (neovim, mpv, cgit, etc.)

if you apt install libluajit-5.1-dev you can compile and link your c programs with it without having to compile it yourself

similarly if you are writing programs in lua rather than in c you can run them in luajit if you just apt install luajit

if you want to release your programs as free software so other people can compile and link them, it's important for them to be able to get the version of luajit you were using so they can debug why your program doesn't work in the current version of luajit

I've never needed to build LuaJIT as a git repo. Just download the tarball and did: make and make install; all I have is GNU make and GCC.

Am I missing something?

The build script changed slightly four days ago, and you will need git in the future (and build from a checkout of the repo.)

That's a shame; I thought it was one of the nice features of LuaJIT. In fact it built its own intermediate tools needed to build LuaJIT dist.

You don't need Git. It just issues a warning if it's not able to infer git information.

It could be using git submodules or tags. That isn't very concerning.

yeah, obviously this isn't acceptable for things like debian, ubuntu, or guix

Here are some performance measurements compared to the previous (2017) release: http://software.rochus-keller.ch/are-we-fast-yet_LuaJIT_2017....

I'm happy to see that the project is still active. LuaJIT is really something special.

I'm also glad that they're using rolling releases, as I felt "bad" for using something named LuaJIT 2.1 beta3 in a project since the target was ARM64.

Huh. I thought this project was abandoned, with the beta 2.1 being the latest version for many years now, and the new GC having never been implemented. But looking at the page now, all mention of this has been deleted. Has work on it resumed? Exciting!

Went to the site and could not find info about Lua version support, is LuaJIT still Lua5.1 only these days? I hope it can support newer versions, e.g. 5.4 etc.

5.1 and a few features from 5.2 such as hexadecimal escapes in strings. I could be woefully out of date but they are/were listed here http://luajit.org/extensions.html

maybe they should change the name

because to me this sounds like hoping linux can support newer versions of minix

or that openbsd can support newer versions of freebsd

The new Lua versions aren't really that worth it, honestly.

I wish people wouldn't do this. It makes it very confusing to know when it makes sense to upgrade, and depending on your luck you'd be getting a version that has a bug fixed in the commit right after, but you wouldn't know because nobody announced a patch release. I get that release engineering adds a lot of overhead but it helps build a lot of trust between the developer and the user.

LuaJIT is in a very weird place overall. Some years ago, the main developer announced that they were stepping down. A bunch of forks sprang up as a result. Meanwhile, the main developer declined to put anyone else in charge of the project, and basically kept developing it on their own (albeit at a slower pace). The forks eventually died; no one was able to build enough momentum to keep a community going.

So this policy was basically where we were anyway. I'm glad the project hasn't died, and in my practical usage it's been very stable, so I haven't had trouble grabbing random development commits off the Git repo. My impression was the release process was very high overhead, and I'd rather that the main developer stick around to work on the code rather than leave due to burning out.

Cloudflare

In case anyone is wondering how highly performant / scalable LuaJIT is - Cloudflare was built on top of LuaJIT for the first ~10 years of it's existence and it's only been in the last couples of years they have begun to use Rust.

https://blog.cloudflare.com/tag/lua/

and they don’t now for good reason. Lua and LuaJIT are terrible to work with.

Your opinion is not universal.

> "LuaJIT is a cool thing. Helped us a lot. But not everything remains the right thing forever."

Posted only 11 months ago by Cloudflare CTO

https://news.ycombinator.com/item?id=32869567

Cloudflare was already operating a scale and as public company, when they were running LuaJIT.

Chances that you'll operate at Cloudflare scale is like 1:Billion.

Sounds like Cloudflare was quite happy with LuaJIT.

Of course they would. It’s called saving face.

They got lucky, choosing a niche language and a massive amount of tech debt as their primary technology stack.

But in the end, Cloudflare is one company. Who gives a what they did?

I don't know about you but a 3x performance increase going from a scripting language with fast iteration times to a programming language that has long compilation times plus a borrow checker is hardly worth the risk of a rewrite to the average company.

Add on top the fact that one of the performance problems was interfacing nginx with luajit, aka there are two unnecessary memory copies.

When I think about these things, I'm thinking cloudflare was lucky enough to outgrow lua rather than being lucky to have gotten away with luajit.

I like Lua the language, but why is it and LuaJIT terrible to work with in your experience?

I haven't looked at LuaJIT, but Lua is written in this weird bastard language using the C preprocessor. All of the terseness of Lisp, with no type system (since it's the C preprocessor), and the added fun of pointer arithmetic.

Would you mind elaborating on this? I'm perusing the source to try to make sense of this comment and all I see is... C.

Literally any header file in https://github.com/lua/lua

In particular, https://github.com/lua/lua/blob/master/ldo.h

Are you referring to Lua, the language?

Or its implementation details (written in C)?

If the later, how does that make Lua, the language, bad?

I'm criticizing the implementation.

I have no complaints about the language (other than arrays starting at 1). I'm just salty because other people have said the implementation was really clean C, so I decided to spend a week porting it to C#, and I gave up. I could not figure out how closures were actually implemented, and I spent a day trying to understand it.

That weird bastard language is called C. It's really easy to hack on the lua codebase, either adding stuff to it or using it as the base for other languages. Provided you can read C, anyway.

Luajit has a really complicated build process. I'm more nervous about changing that codebase.

I would be more concerned about the high percentage of assembler than the build process.

I used to have problems with luajit. I tried it out right now and it works. Even the ffi is working properly without having to resort to a janky GitHub fork (that one doesn't work on ARM for me).

Would be nice to have support for S390 finished and merged. There are some PRs but they aren't ready yet.

Interesting, the Debian packages ship with s390x support.

Reply on news.ycombinator.com