Testing the Lua kernel interpreter in NetBSD (2021) | Hacker News Reader

Testing the Lua kernel interpreter in NetBSD (2021)

unitedbsd.com

80 pointsnetworked24 comments

Very cool to see this over here :) A past GSoC project of mine ported the entire Lua test suite to the NetBSD kernel space: https://github.com/gszr/luatests (it needs to be updated... it's been on my to-do list). See also https://github.com/gszr/luaio for some cool IO bindings for Lua in kernel.

This sounds like a really cool feature, but given that I'm not an expert I'm not sure what the practical use-cases for this are. Is it similar to what you can do with something like eBPF in Linux?

Yes, although their scopes are different. For Linux it started as a simple virtual machine for programmable firewall rules. For NetBSD, it's more of a playground to try ideas in kernel space, while rump kernels allow to try kernel ideas in user space.

What are the limitations in terms of what the interpreter has access to? Can it interact with devices, physical memory, etc.?

It has its own memory namespace, so it won't be allowed to access the kernel memory. It has however full access to the devices.

The /dev/lua device, which allows interaction with the kernel interpreter: https://man.netbsd.org/lua.4

Presentation: https://www.netbsd.org/gallery/presentations/mbalmer/fosdem2...

But what is the practical gain from moving it to kernel-space? Could have at least compared the execution speed against running it normally on user-space to see whether there is any speed-up factor.

I don't think it was ever about speed; it's about instrumenting things in kernel space.

And for what reason to choose lua over using languages more matured on system programming like Rust..?

I expect it's because it's an interpreted, scripting style language. Doesn't require a compile step, is meant to be ad-hoc evaluated, etc.

Also the Lua runtime is very small and its implementation readable, simple, and easily embedded.

I think of Lua as kind of filling the niche that BASIC did in the 80s.

Then why not BASIC ?

Not sure if you're being serious or if this is one of those silly, "why not Rust" responses that are becoming a trope...

Read the article, and you'll see the most glaring reason: Rust doesn't have an interpreter. Also, Rust isn't very portable, so that'd exclude a number of platforms on which NetBSD runs. Add that Rust is not lightweight in the least, and you can see why not Rust.

Also, Lua is much, much more relevant for uses like this, and suggesting that Rust is more mature is just... well, wrong. Lua has been around for ages and has matured quite well.

> Not sure if you're being serious or if this is one of those silly, "why not Rust" responses that are becoming a trope...

I am being serious, and no I never used Rust in my life, but I do know many OSes trust it and are gradually adopting it

> Read the article, and you'll see the most glaring reason: Rust doesn't have an interpreter.

Are you sure we are pointing to the same article? Where does it say anything about that at all?

for the entire small article, the author just plays around with a kernel module that implements lua interpreter somehow with no context presented at all by the author of why there is a kernel module for lua in the netbsd, and that is my doubt.

why is it there rather than being used normally on user-space? what is netbsd goal with integrating lua on the kernel? And so far nobody has answered me properly.

> Also, Lua is much, much more relevant for uses like this, and suggesting that Rust is more mature is just... well, wrong. Lua has been around for ages and has matured quite well.

Mention me a _serious_ operating system using lua on their kernel side and I'll acknowledge that last sentence.

Be sure we are talking on the same page, we are talking on the context of being used for programming operating systems, is lua mature for that domain?

> Mention me a _serious_ operating system using lua on their kernel side and I'll acknowledge that last sentence.

NetBSD, of course.

> Be sure we are talking on the same page, we are talking on the context of being used for programming operating systems, is lua mature for that domain?

As https://www.lua.org/about.html notes,

> Lua is embeddable

Lua is specifically designed to be embedded into other things in order to make them easily extensible.

Lua is pretty popular for scripting in video games

Why wouldnt it be mature for OSes

> And for what reason to choose lua over using languages more matured on system programming like Rust..?

...you wanna talk about mature? Rust hit 1.0 in 2015 (https://blog.rust-lang.org/2015/05/15/Rust-1.0.html). Lua hit 1.0 in either 1993 or 94 (https://www.lua.org/history.html is a little fuzzy). As for reasons why Lua is suitable, and Rust is not even remotely suitable for anything core to NetBSD, the obvious one is platform support; NetBSD supports basically every platform to ever be capable of running a unix-like - https://netbsd.org/ports/ has a list. Rust supports... some of these, yes, but only a tiny fraction. Thus, Rust is wholly unsuitable (at this time) for any core component of NetBSD.

Interesting! Is this the answer to Linux EBPF?

I wouldn't call it an answer. In-kernel Lua states[1] have been released with NetBSD 7.0 in 2015 and the work started at least as early as 2013, probably 2011.

See https://news.ycombinator.com/item?id=6562611 for a previous discussion.

Edit:See [2] for a presentation at FOSDEM 2012

[1] https://man.netbsd.org/lua.4 [2] https://www.netbsd.org/gallery/presentations/mbalmer/fosdem2...

There is also `secmodel_sandbox`, a proof of concept for a Lua-based application sandbox on NetBSD:

https://github.com/smherwig/netbsd-sandbox

I really wish it was picked up more. NetBSD has a lot of innovative ideas and it is sad to see it go so forgotten and neglected.

NetBSD (and pkgsrc) have a very loyal user base, with quite different expectations from their system than the Linux or FreeBSD user base.

I guess what NetBSD is after (preserving the heritage of the original / early Unixes from which it descends, keeping things simple and stable with minimal requirements while staying as platform agnostic as possible) is a bit of a niche, which also explains the small user base.

More users favor simplicity of usage rather than technical simplicity, run powerful aarch64/amd64 machines rather than exotic / constrained hardware, and are more after the "move fast" model than after long term technical stability and consistency. A large "frenetic" project like Linux simply suits those users better.

Last I heard ePBF is basically stalled due to Spectre. They aren't planning to expand it. Interesting to see NetBSD doing that.

eBPF is being eagerly and actively expanded; just look at [1]. What's stalled is unprivileged eBPF, i.e. using it without being root.

[1] https://lwn.net/SubscriberLink/922405/d85450150ef64840/

Ah yes, I meant unprivileged eBPF. Although your link doesn't really support your statement given the only review comment!

If only BSD had something like bpf... /s

UnitedBSD is great if you need a little help with your netBSD install, or just want to interact with other BSD/unix types. Very friendly community.

The author of the linked post, JuvenalUrbino, contributes a great deal imo.

Reply on news.ycombinator.com