Show HN: LuaJIT assembler/assembly language for direct bytecode programming | Hacker News Reader

Show HN: LuaJIT assembler/assembly language for direct bytecode programming

github.com

141 pointsRochus29 comments

Thanks for the suggestion. I think mostly people who want to use LuaJIT as a backend for another language, not primarily Lua programmers, might be interested in my work. Here is an article describing the project: https://medium.com/@rochus.keller/implementing-call-by-refer...

Perfect, I will include it in the next issue.

> LuaJIT as a backend for another language, not primarily Lua programmers

Not exactly backend but that kinda fits my use case. I have a C embedded application that I need scripting on. Nothing unsecure from the user, from my own backend, but it requires a foreign function interface, and its a super constrained device.

Lua, JavaScript, forth, etc. None I’ve been able to find anything I like under 50 kB flash or are even slightly “friendly”.

Found one in JavaScript was close, but it’s an interpreted language which means every letter of code is a byte I need to transfer then have parsed.

I’ll look into this. Esp if it’s still possible to send just bytecode in.

Normally JavaScript is a JIT compiled language because of the need to do stuff like hoisting etc.

Just a nit on the JIT... JavaScript is almost always compiled, regardless of whether there is a JIT or not.

A typical JavaScript system first compiles the JS source code into bytecode. Then it starts executing (interpreting) the bytecode.

If there is a JIT, that is a second compiler. It monitors the execution of the bytecode to find hot spots where it compiles the bytecode into machine language.

yeah but I was just nitpicking on the "interpreted" statement above. What comes around goes around I guess.

I don't know if luajit runs in your device, but vanilla luajit supports generating and loading bytecode.

https://luajit.org/running.html

I think primarily it’s the ffi I would be missing. This would get me a turing complete language but set up as an executable it’s wrong for me.

Notice that if I remember correctly, you can generate bytecode on the PC (by using the luajit command) and have the resulting bytecode loaded through the C API on your device as if it were a regular script.

The basic lua VM (not luajit) is only like 11kb compiled. You can't directly do FFI from lua, but you can set up the FFI functions from the C API

Any examples of code or know where someone has done this?

Lua was made for this! Here's a simple example:

https://gist.github.com/Vesnica/3672425

It loads and runs a Lua source code file which calls the exported C function. Lua compiles the source code into byte code before running it but that can also be done ahead of time with the Lua compiler:

https://www.lua.org/manual/5.3/luac.html

It can also be done manually with the dump and load functions:

https://www.lua.org/manual/5.3/manual.html#lua_dump

https://www.lua.org/manual/5.3/manual.html#lua_load

I don't think this work fits your needs. Have a look at something like https://en.wikipedia.org/wiki/NodeMCU or http://www.eluaproject.net/ or https://github.com/whitecatboard/Lua-RTOS-ESP32 instead.

The problem with eLua and others is that they either try and replace C or they focus on the headers/registers/peripherals of the micro. Design it to be the environment you do all your work in. Get there, stay there. It’s why eLua appears to have terrible micro support because they only have the bandwidth for specific chips. Other options like MicroPython have a VM you are never supposed to leave.

I think you’re right that Lua is the way to go. But I have yet to see someone package it nicely for gcc and to be included and built from an MCU IDE (Keil, Segger Emebbeded, IAR, etc). I’ll keep looking.

You are missing https://libraries.io/github/fperrad/tvmjit which adds a nice macro and OO layer (in lisp syntax) on top of LuaJIT. A much nicer way to use this superior VM.

404 on all GitHub links

Yeah, he left github. Maybe you'll get it through shine, which uses tvm. https://github.com/richardhundt/shine

Or I'll upload a fork.

Here you are: https://github.com/perl11/tvmjit From an older richardhundt/shine source, in its deps.

Found it also at https://github.com/richardhundt/tvmjit

Thanks, interesting. But so far I don't see such a big advantage over LuaJIT or the Lua language that would justify creating a new, incompatible fork. It's just another variant, not necessarily a better one. Or have I missed something which is so much better than the original?

Is RaptorJIT abandoned? It seems like they got a couple optimizations, deleted most of the dynasm backends, then gave up.

RaptorJIT is working well for Snabb but it's not clear what other projects can benefit from it. We're using Lua to compete with C while most other projects seem to be using it to compete with Go/Python/node. So for the moment I'm not actively courting new users because I don't find many kindred spirits in the wider LuaJIT community.

EDIT: To clarify, the big advantage of RaptorJIT is that it takes dramatically less time and effort to profile and optimize programs to become competitive with C. If you're spending a lot of time scratching your head over long 'jit.dump' reports then RaptorJIT will save you lots of time. That's a big win for Snabb hackers but we seem to spend much more time on JIT-optimization than other LuaJIT users.

(Anecdotally it seems like most other LuaJIT users eventually give up on understanding the JIT entirely because they don't see it as being worth the effort. This doesn't really bode well for LuaJIT world domination.)

I had also a look at RaptorJIT. But as far as I understood it only runs on x86-64, not on x86. Or do I have a misunderstanding?

That's correct.

Does your fork base on LJ 2.0 or 2.1 beta?

I really like the streams and resources you put up, Luke. Thanks for clarifying. Studio is so cool, I've been looking at maybe hacking it to work with my spreadsheet compiler, using Lua (or maybe LJ bytecode) as IR, and source maps.

Your live streams are a massive resource.

They are so dazzling to others that they'll sit through an explanation of phi nodes, while I assure you they would not otherwise. If you ever want a change of pace, we'd probably be interested in bringing you on board in 2020. Cheers, and I hope the fellas are doing well. :- )

Would you be interested in writing a short editorial on this for the Lua Digest newsletter?

https://luadigest.netlify.app

This is very interesting work! I am poking around CPython and Python bytecode to learn about compilation/VM/interpretation and I was thinking about writing cose directly in Python bytecode to play with.

The UI on top looks very handy!

Kudos

Tangential, but what's LuaJIT's maintenance status?

I was also interested, here's the latest update: https://www.freelists.org/post/luajit/Regular-releases-for-l...

It doesn't look like there's been a release since May 2017 though.

Reply on news.ycombinator.com