Scriptable operating systems with Lua (2014) [pdf] | Hacker News Reader

Scriptable operating systems with Lua (2014) [pdf]

netbsd.org

71 pointstype025 comments

Seems like a perfect use-case for Janet. (https://janet-lang.org/) A fast minimal VM like Lua, but even more extensible than Lua by being a "Lisp" with macro and C extension capabilities. Not a true Lisp, it's very pragmatic and performance-oriented. But it keeps the good stuff. Composing OS extensions with something like that would be downright joyous.

A lovely idea. I'd been trying to find an excuse to mess with Janet, because so many of the decisions seem nice, but hadn't figured out where it wanted to be. That idea sounds compelling, though!

I know this is a highly subjective and controversial opinion, but to me this syntax looks even worse than Rust, which I already can't read.

You must be fairly new to the wider language space. Admittedly, the example Janet presents on their frontpage doesn't at all demonstrate the strengths of S-expression syntax. I highly recommend taking a look at Clojure or Racket to get a sense of what Lisp syntax can do. A lot of what you'll find would be very easily portable to Janet.

Actually I have seen many Lisp-style languages and I do not like any of them, I simply cannot stand the syntax.

What a silly reason to be afraid of Lisp. The simple s-expression syntax is the source of Lisp-like language's homoiconic power, and the syntax of most other languages is just as terrible if not worse, but you're just so used to it you don't notice.

If you're a professional programmer, then you regularly put up with vastly worse syntax than Lisp every day without realizing it, I bet, like bash nonsense, or yaml and its templating languages, or Perl acoustic coupler modem line noise, or C preprocessor macros, or C++ and its delightful templates, or TypeScript shape declarations, for example (I could go on and on naming popular languages with terrible syntax, but you get my point).

If you don't understand the power and uses of homoiconicity and metaprogramming, then of course you're not going to understand or appreciate why Lisp has that syntax.

PostScript is also homoiconic and jsonically polymorphic (i.e. it has essentially the same data types as JSON, just decades earlier), but if you can't deal with Lisp's parenthetical syntax, then PostScript's reverse polish notation syntax will probably be too much for you too.

The Shape of PSIBER Space: PostScript Interactive Bug Eradication Routines — October 1989:

https://donhopkins.medium.com/the-shape-of-psiber-space-octo...

Homoiconicity:

https://en.wikipedia.org/wiki/Homoiconicity

>In computer programming, homoiconicity (from the Greek words homo- meaning "the same" and icon meaning "representation") is a property of some programming languages. A language is homoiconic if a program written in it can be manipulated as data using the language.[1] The program's internal representation can thus be inferred just by reading the program itself. This property is often summarized by saying that the language treats code as data.

>In a homoiconic language, the primary representation of programs is also a data structure in a primitive type of the language itself.[1] This makes metaprogramming easier than in a language without this property: reflection in the language (examining the program's entities at runtime) depends on a single, homogeneous structure, and it does not have to handle several different structures that would appear in a complex syntax. Homoiconic languages typically include full support of syntactic macros, allowing the programmer to express transformations of programs in a concise way.

>A commonly cited example is Lisp, which was created to allow for easy list manipulations and where the structure is given by S-expressions that take the form of nested lists, and can be manipulated by other Lisp code.[2] Other examples are the programming languages Clojure (a contemporary dialect of Lisp), Rebol (also its successor Red), Refal, Prolog, and possibly Julia (see the section “Implementation methods” for more details).

I'm not afraid of it, I just don't like the way it looks, I don't like looking at it.

I also don't appreciate this "you're just not able to appreciate it" false flag.

It doesn't matter to me if another language magically solved all the problems I'll ever have 10x easier, if I can't stand to look at it, it's a non-starter for me.

Your loss.

They embedded Lua into the NetBSD kernel for scripting frequency scaling and network packet filtering. Their packet filtering example ran a 100Mbps connection at 96Mbps, which indicates the performance hit isn't extreme.

Would probably need a few more benchmarks to determine if it scales or if it's a bottleneck

This work ran more benchmarks with NPFLua:

https://2018.eurobsdcon.org/static/slides/Fast,%20Flexible%2...

We have also integrated Lua with Netfilter and XDP on Linux, filtering up to 5.3 Gbps.

https://netdevconf.info/0x14/session.html?talk-linux-network...

It’s just undertaker that lua is such a quirky language. A few things different and it would be better.

1 vs 0 index, tables and arrays. I don’t remember the rest, but those were super annoying 15 years ago

If interested, you may want to glance at https://github.com/luainkernel/lunatik

Discussed at the time:

Scriptable Operating Systems with Lua [pdf] - https://news.ycombinator.com/item?id=8389728 - Sept 2014 (50 comments)

Oh my god, somebody actually remembered MaxScript a decade ago! That makes my day.

https://news.ycombinator.com/item?id=8391583

I had a blast with MaxScript -- it was essentially a re-implementation of Kaleida Lab's ScriptX, which itself was like an object oriented Scheme without (so many of) the parens but with infix notation and a great plugin extension system. No co-incidentally they were both designed and implemented by the same person, John Wainwright.

https://news.ycombinator.com/item?id=23424848

I recently used embedded Lua within Nginx to alter requests on a reverse proxy using Lua scripting.

It's great that Lua can be seamlessly integrated wherever we need it, allowing for the extensions and implementation of custom logic. It's also popular in gaming and often used to develop different custom game mods.

IIRC, Lua in Nginx is deprecated in favor of the NodeJS module. I see more and more Nginx extensions written in JS, for SAML authentication, HTTP ACME challenge, etc...

Do you mean njs? Or is there a really a NodeJS module?

https://nginx.org/en/docs/njs/

Yes, njs. My dumb mind expanded the "n" to "node".

It's maybe deprecated by the official Nginx support, but there are other projects and organizations that are offering Lua scripting with Nginx with all kinds of extensions and libraries.

See OpenResty website[0] and Github repo[1].

[0] - https://openresty.org/en/

[1] - https://github.com/openresty/

Putting a dynamic interpreter into the kernel is an old idea, that Sun did with PostScript 34 years ago, for NFS 3, aka "NeFS", inspired by the architecture of the NeWS window system.

Another advantage is that network clients can efficiently push the computing to the file server, avoiding millions of context switches and network round trips, by sending programs to perform intricate repetitive operations directly in the kernel, instead of performing each individual operation over the network.

https://news.ycombinator.com/item?id=39762839

DonHopkins 33 days ago | parent | context | favorite | on: Understanding Google's File System (2020)

It got lost a lot longer than 10 years ago. Sun published the NFS 3 spec (NeFS -- Network extensible File System) 34 years ago in 1990, which was in turn based on John Warnock's earlier ideas of using PostScript as a "linguistic motherboard" with "slots" for different kinds of "cards" like graphics and file systems (drawing from his experience at Adobe and Xerox PARC):

        PostScript is a linguistic "mother board", which has "slots"
        for several "cards".  The first card we (Adobe) built was a
        graphics card.  We're considering other cards.  In particular,
        we've thought about other network services, such as a file
        server card.

https://news.ycombinator.com/item?id=34256293

[...]

DonHopkins on Sept 5, 2019 | parent | context | favorite | on: Samsung Announces Key-Value SSD Prototype

It's like NFS 3 (NeFS) in hardware! (NeWS for disks: A PostScript interpreter in the kernel as a file system API.)

https://www.donhopkins.com/home/nfs3_0.pdf

Network Extensible File System Protocol Specification (2/12/90)

Comments to: sun!nfs3 nfs3@SUN.COM

Sun Microsystems, Inc. 2550 Garcia Ave. Mountain View, CA 94043

1.0 Introduction

The Network Extensible File System protocol (NeFS) provides transparent remote access to shared file systems over networks. The NeFS protocol is designed to be machine, operating system, network architecture, and transport protocol independent. This document is the draft specification for the protocol. It will remain in draft form during a period of public review. Italicized comments in the document are intended to present the rationale behind elements of the design and to raise questions where there are doubts. Comments and suggestions on this draft specification are most welcome.

[...]

Although it has features in common with NFS, NeFS is a radical departure from NFS. The NFS protocol is built according to a Remote Procedure Call model (RPC) where filesystem operations are mapped across the network as remote procedure calls. The NeFS protocol abandons this model in favor of an interpretive model in which the filesystem operations become operators in an interpreted language. Clients send their requests to the server as programs to be interpreted. Execution of the request by the server’s interpreter results in the filesystem operations being invoked and results returned to the client. Using the interpretive model, filesystem operations can be defined more simply. Clients can build arbitrarily complex requests from these simple operations.

[...]

https://news.ycombinator.com/item?id=22456710

DonHopkins on March 1, 2020 | parent | context | favorite | on: Sun's NeWS was a mistake, as are all toolkit-in-se...

Owen Densmore recounted John Warnock's idea that PostScript was actually a "linguistic motherboard". (This was part of a discussion with Owen about NeFS, which was a proposal for the next version of NFS to run a PostScript interpreter in the kernel. More about that here:)

https://news.ycombinator.com/item?id=17077721

Owen Densmore's discussion of John Warnock's "Linguistic Motherboard" idea for PostScript:

https://donhopkins.com/home/archive/NeWS/linguistic-motherbo...

    Date: Tue, 20 Feb 90 15:20:52 PST
    From: owen@Sun.COM (Owen Densmore)
    To: don@cs.UMD.EDU
    Subject: Re:  NeFS

    > They changed the meaning of some of the standard PostScript operators,
    > like read and write, which I don't think was a good idea, for several
    > reasons... They should have used different names, or at least made ..

    Agreed.  And I DO see reasons for the old operators.  They could
    be optimized as a local cache for NeFS to use in its own calcs.

    > Basically, NeFS is a *particular* application of an abstraction of
    > NeWS. The abstract idea is that of having a server with a dynamically
    > extensible interpreter as an interface to whatever library or resource
    > you want to make available over the network (let's call it a generic
    > Ne* server).

    Very true.  This has been particularly difficult for me to get across
    to others here at Sun.  I recently wrote it up for Steve MacKay and
    include it at the end of the message.

    > It's not clear to me if NeFS supports multiple light weight PostScript
    > processes like NeWS.

    I asked Brent about this, and he agreed that it's an issue.  Brent
    has been talking to a guy here who's interested in re-writing the
    NeWS interpreter to be much easier to program and debug.  I'd love
    to see them come up with a NeWS Core that could be used as a generic
    NetWare core.

    I think you should send  your comments off to nfs3 & see what happens!
    I agree with most of your points.

    Owen

    Here's the memo I consed up for MacKay:

https://news.ycombinator.com/item?id=22456710

Thanks for sharing this! Is it open source?

Lua - underestimated

Years ago, there was a project for Linux (now defunct) that used Scheme to script the Linux kernel.

http://www.abstractnonsense.com/schemix/

Reply on news.ycombinator.com