• Do any interpreters/compilers make use of this functionality? I know D has fibers, and Java is planning on adding them.
• How does it compare against 'green threads'?
• Can't this kind of thing be done without making syscalls into the kernel?
> The LuaJIT VM is fully resumable. This means you can yield from a coroutine even across contexts, where this would not possible with the standard Lua 5.1 VM: e.g. you can yield across pcall() and xpcall(), across iterators and across metamethods.
This implies that if a Lua function calls a C function that C function acts like it's on the stack of the current Lua coroutine and can yield (suspend itself and the coroutine) and be resumed. Note that PUC Lua 5.2 added a different (more portable) mechanism for accomplishing the C-lua part of this: lua_yieldk, lua_callk, and lua_pcallk [3], which require the programmer to do all the hard work themselves. LuaJIT just does it by magic.
That PUC Lua and LuaJIT have suspendable C functions is something that IMO sets them apart as truly mature scripting language implementations; almost no other scripting languages can do this!
[1] https://coco.luajit.org/portability.html [2] https://luajit.org/extensions.html [3] https://www.lua.org/manual/5.2/manual.html#4.7
Cooperative threads like this let you completely avoid having to develop state machines for each cycle within state machines for each instruction, etc. They let you suspend a thread four levels into the call stack, and then immediately resume at that point once other emulated processors have caught up to it in time. That lets you do fun tricks like only synchronizing components when required, so it can in some instances end up not only far more elegant, but also much faster than state machines, when they're used well.
I wrote a bit more about this and showed some examples here if anyone's interested: https://near.sh/articles/design/cooperative-threading
I also use them for my web server because I like them, but there are probably better ways of doing that.
[0]: https://docs.microsoft.com/en-us/sql/database-engine/configu...
https://github.com/socketry/async https://github.com/digital-fabric/polyphony
Disclaimer: I'm the author of Polyphony.
I imagine you've already seen [1] and the two other articles it links to.
The main difference, at least from my limited view, is that D fibers have to explicitly yield and as far as I know in project loom they've built this yielding into the JVM when I/O occurs. It looks like libraries have to intercept libc calls in D to achieve the same thing [0].
I also see that D provides a fiber scheduler, but it only schedules on a single thread vs loom's virtualThreadExecutor which will use multiple threads.