Piccolo – A Stackless Lua Interpreter
kyju.org
kyju.org
Tons of awesome technical details in the post and I've always felt that Lua's co-routines are incredibly undervalued. We used to run Lua in all sorts of crazy places and it really punches above its weight.
The above quote + zoom out at the end also makes some really good observations. Some of the most powerful systems I've worked on picked a couple core technologies that were complementary with good interop rather than a one-size fits all. Will be interesting to see where Piccolo goes.
The issue is that there exist ways to make a single bytecode instruction take an unbounded amount of time to execute. For example, Lua's "string.find()" is implemented in native code. The interpreter only sees a single OP_CALL opcode, so it will count it as 1 instruction executed. But the actual execution time of the native implementation of string.find() is dependent on its inputs, which are not only variable length strings, but can be maliciously crafted to run in exponential time. Here's an example, shamelessly stolen from Mike Pall himself:
string.find(string.rep("a", 50), string.rep("a?", 50)..string.rep("a", 50))
You will find that this is precisely the approach Piccolo takes.
Piccolo looks amazing, I got a perfect use case for it, and I'm excited for when I get the chance to use it, thank you for working on it.
Is this library offering a lua implementation more well-designed for this use-case? I got all this code to unload the coroutine stack to store and and reload it before continuing it later. Does having C bindings to this library makes sense?
My understanding is that the "stackless" concept here means that it does not store its execution state in the "C runtime stack" (or Rust, in this case).
So, there is some blob of memory describing that info, managed by Piccolo, rather than it residing on the "real" OS execution stack.
In particular, for call chains like: Lua -> C -> Lua -> C (or so), it is normally hard to save/restore the "C" parts of that chain, since you need to peek into the not-normally-accessible and platform-specific C runtime stack details. I wonder: how are you doing it in your system?
In Piccolo, I imagine it would be easier, since even the "-> C ->" portions of the stack are being managed by Piccolo, not the base C runtime. I don't know what the APIs to access that stuff actually look like, though.
Aside: have you looked at Pluto/Eris for Lua serialization of coroutines?
----
EDIT Yes, it seems like the section The "Big Lie" is right up your alley (:
>You should exert care when using this library. Several of its functions violate basic assumptions about Lua code (e.g., that variables local to a function cannot be accessed from outside; that userdata metatables cannot be changed by Lua code; that Lua programs do not crash) and therefore can compromise otherwise secure code (from the manual)
kinda creeps me out.
Please don’t use monospaced fonts for long-form text.
It’s extremely different to read because your eye struggles to see the “shape” of each word, when everything is equally spaced apart.
Also, as an European I used the read the Teletext (It was almost an electronic newspaper with national/international news/sport results/weather forecasts and so on which worked accesing every page/section by a 3 digit number) on a CRT TV just fine.