Lunatik: Lunatik is a framework for scripting the Linux kernel with Lua | Hacker News Reader

Lunatik: Lunatik is a framework for scripting the Linux kernel with Lua

github.com

186 pointstodsacerdoti36 comments

Open articleView on HN

Happy to see this on HN =). Lunatik’s main author here. AMA.

Please feel welcome to join us on Matrix [1] as well.

[1] https://matrix.to/#/#lunatik:matrix.org

What are some of the main use cases for this project?

the most explored use case is for scripting networking, please see:

[1] https://legacy.netdevconf.info/0x14/session.html?talk-linux-... [2] https://netdevconf.info/0x17/sessions/talk/scripting-the-lin... [3] https://www.netbsd.org/~lneto/dls14.pdf [4] https://www.slideshare.net/eurobsdcon/lneto-npf-scripting [5] https://2018.eurobsdcon.org/static/slides/Fast,%20Flexible%2...

However, it was conceived for extensibility and flexibility in general, in the spirit of [6]. It has been used also for scripting CPU frequency scaling, debugging, application sandboxing among others [7,8].

[6] https://web.stanford.edu/~ouster/cgi-bin/papers/scripting.pd... [7] https://drive.google.com/file/d/1TJ50IEbRXlBz-TxheGdkO-b4vLS... [8] https://youtu.be/-ufBgy044HI?si=IT4FOqeldA-jsg2r (Portuguese only)

ZFS also has its own kernel Lua version (I'd love to unify them, BTW).

Moreover, we have some simplified examples on our repository [9].

[9] https://github.com/luainkernel/lunatik?tab=readme-ov-file#ex...

Could this allow for a hard real time Lua programmable system if you replace gc?

Potentially, yes. However, I didn't try it. You could even disable GC completely. Lunatik provides some facilities that could be useful in this scenario as well, such as RCU tables and shared objects. For instance, you could pre-allocate the entire memory to be used by your Lua states and treat them as volatile. In this case, you could persist state on the shared objects stored and synchronized through a RCU table. Of course, you could implement your own GC or tune the existent GC modes as well.

Does this use or relate to eBPF in any way?

It doesn't use eBPF but we can interoperate.. we have developed XDPLua for this as a GSoC project [1] and then published this work on Netdev 0x14 [2].

[1] https://victornogueirario.github.io/xdplua/ [2] https://netdevconf.info/0x14/session.html?talk-linux-network...

I'm currently working on updating XDPLua for using kfuncs on top of the newest version of Lunatik.. I've a PoC already and should create a PR in a few weeks.

Reminds me of Lua in the NetBSD kernel

https://archive.fosdem.org/2013/schedule/event/lua_in_the_ne...

Edit: It made it to HN front page today.

it's totally related.. we started with Lunatik in Linux actually then ported it to NetBSD..

[1] https://www.netbsd.org/~lneto/dls14.pdf

a bit of history.. https://mail-index.netbsd.org/tech-kern/2010/10/05/msg008900...

https://news.ycombinator.com/item?id=40086027

There was something like this using Scheme, maybe 20 years ago, but I don't remember any details and don't think I ever tried it.

Edit: see http://www.abstractnonsense.com/schemix/

Interestingly, this project (Lunatik) started over 30 years ago in 1993.

https://github.com/luainkernel/lunatik/graphs/contributors

this is because the repository is a Lua fork.. actually, Lunatik started in 2008 [1, 2].

[1] https://sourceforge.net/projects/lunatik/files/ [2] https://www.maxwell.vrac.puc-rio.br/colecao.php?strSecao=res... (Portuguese only)

Ok, makes sense. Thanks for clarifying

Thank you for your efforts. This is a really good project. Kudos

Does this finally allow to write firewall rules in a sane and flexible language?

Please take a look at:

[1] https://netdevconf.info/0x14/session.html?talk-linux-network... [2] https://netdevconf.info/0x17/sessions/talk/scripting-the-lin...

I'm currently updating XDPLua and we might have a GSoC project on updating NFLua this year =)..

[3] http://www.lua.inf.puc-rio.br/gsoc/ideas2024.html#lunatik-ne...

> Lunatik does not support floating-point arithmetic

Assuming there is one, what's the technical reason for this?

> Kernel code is normally prohibited from using floating-point (FP) registers or instructions, including the C float and double data types. This rule reduces system call overhead, because the kernel does not need to save and restore the userspace floating-point register state.

> However, occasionally drivers or library functions may need to include FP code. This is supported by isolating the functions containing FP code to a separate translation unit (a separate source file), and saving/restoring the FP register state around calls to those functions. This creates “critical sections” of floating-point usage.

https://www.kernel.org/doc/html/next/core-api/floating-point...

This is so beautiful I could almost weep.

With this, we can proceed - easily! - to attain something I have long wanted: to replace all Linux userspace with Lua-only tooling. Yes, Lua all the things in userspace.

I know, "why would I do that?", well, because there is a thing I've wanted for 40 years of computing, and that is a very, very powerful computer that only requires me to use one (oh, okay, two or three..) languages.

I grew up in computers that booted directly to a single language (BASIC), and would let you do everything you need in that and only that. Imagine replacing all userspace tooling with a lua implementation. Yes, insane! But imagine the development flow once you get some good stuff ported/ffi'ed! Having the kernel interfaces available now, is just .. candy.

I would love to have this for a Linux distro, even experimental, even just as a toy, to boot directly into a LOAD81[1] interface that has a whole lotta new Lunatik in it .. I could imagine pushing LOAD81 very hard into a new OS paradigm with this. Wow!

[1] = antirez' LOAD81, which I have long thought of as a fun way to build a new userspace front-end: https://github.com/antirez/LOAD81.git

One of the main inspirations for Lunatik is the MS Singularity OS [1], which gets rid of kernel/user space division and implements application/process protection by software using programing language resources. The other one is the MIT Exokernel [2], which provides a tiny kernel with almost no abstractions responsible only for safely multiplexing the hardware. Now, imagine a tiny kernel written in C, exposing Lua APIs, responsible mainly for multiplexing HW; and user processes implemented as separate Lua states running on the same address space, protected by language constructs.

[1] https://pdos.csail.mit.edu/6.828/2008/readings/hunt07singula... [2] https://www.classes.cs.uchicago.edu/archive/2019/winter/3310...

https://imgflip.com/i/8njfrr

I've just glanced at LOAD81 now.. pretty good job! thanks for sharing.. I noted your keyboard event handling (using SDL).. we have something very related to this in Lunatik (at the kernel level, of course) [1]. We've a kinda fun script that uses keyboard event handling to implement a "key locker" using Konami code [2].

[1] https://github.com/luainkernel/lunatik?tab=readme-ov-file#no... [2] https://github.com/luainkernel/lunatik?tab=readme-ov-file#ke...

I didn't write LOAD81 - antirez, of redis fame, did! I'm just a huge fan...

But thanks for the further insight into Lunatik - I'm definitely going to be putting it through its paces soon.

The Arcan project[1] has pushed for this for a loooong time and doing some really interesting cutting edge things with it.

Depending on which set of Lua scripts it runs it fills a different role. The first set becomes the display server and window manager. It can then run itself recursively to provide graphical 'apps' (but also X11 ones, Wayland ones, ...).

I run the 'regular' desktop (stacking, tiling, ...) 'durden' right now, but I hope to switch to the more crazy dataflow/zui one (pipeworld[2]) or the VR one (safespaces[3]).

Then it has an additional 'curses like' one for TUIs and CLIs, 'arcan-tui' with a default shell Lash#Cat9[4]. Looking at their recent posts there also seem to be a network story including server-side processing, but I haven't had the time to play with out more than following the examples in the blog so can't say more about that.

EltaninOS[5] is trying to build a distribution around it.

Everything scripted in Lua.

[1] = https://arcan-fe.com [2] = https://arcan-fe.com/2021/04/12/introducing-pipeworld/ [3] = https://arcan-fe.com/2018/03/29/safespaces-an-open-source-vr... [4] = https://arcan-fe.com/2022/10/15/whipping-up-a-new-shell-lash... [5] = https://eltaninos.org/

And while we're at it, let's run the lua interpreter in ring 0 and really get the party going.

Why lua though?

Please refer to Section 3 from [1], this was extensively discussed there.

[1] https://www.netbsd.org/~lneto/dls14.pdf

However, giving you a short answer, we use Lua mainly because it's easy both to extend and to embed. It's implemented as an almost "freestanding" C library. Moreover, it has a considerable small footprint (~250 KB) in comparison to other scripting languages (e.g., Python has a few MB). Moreover, Lua has automatic memory management, fully isolated execution states, protected calls, among other features.

Sounds like he needs to be introduced to Forth.

Lua has much nicer syntax and much higher abstractions than forth, and is similar tiny

Maybe. Debatable. But the point was more that “replace all userspace [and kernelspace] with X” is the raison d'être of many Forths. So many classic Forth systems boot into a Forth environment, extended in Forth by the programmer-user. If that's what he's looking for, there's 55 years of Forth history to delve into.

Weird miss to not call it Lunatix.

Perhaps when we have a fully scriptable operating system, we might call it Lunatix =).

Reply on news.ycombinator.com