Launch HN: Lunatic (YC W21) – An Erlang Inspired WebAssembly Platform
The two of us met in high school, studied computer science together, but then went separate ways working as backend engineers. Bernard worked at CERN and Hrvoje co-founded Amodo, an insurance tech startup. Bernard, being a huge fan of Erlang/Elixir, started working on a similar open-source runtime for WebAssembly which he called Lunatic (https://github.com/lunatic-solutions/lunatic).
Lunatic runs Wasm modules as lightweight processes with its own heap/stack and preemptively schedules them on a multi-threaded executor. You can spawn those processes using a library we provide (currently for Rust and AssemblyScript) to enable actor-based architectures with message passing. Scheduling is implemented by modifying a Wasm module and adding “reduction counts” (similar to Erlang). You can write seemingly blocking code but it won’t actually block the underlying OS thread as our implementation of WebAssembly System Interface (WASI, think of it as POSIX syscalls) will be implemented with async Rust and a bit of magic [0] :) The code is JIT’ed and we build on top of existing Wasm runtimes Wasmtime/Wasmer for this part (codegen is done by LLVM or Cranelift).
To step back from technical details, working on Lunatic for the past few months, we have started to form a bigger picture about server-side applications. Over the years we have witnessed many trends: Docker & containers became popular, asynchronous programming and green threads are ubiquitous for IO intense work, polyglot codebases are always present, microservice architecture became popular, and distributed is being used both for scale and to bring computation closer to clients to lower latency. Two important driving forces are hardware capabilities and how we develop software. Those have changed dramatically from the time operating systems were created.
For example, to maximize resource usage of a single machine, we started running virtual machines and then moved to more lightweight containers, both to give isolation and sandboxing to different applications. Serverless is pushing this even further. Lunatic builds on top of WebAssembly security principles [1] to give sandboxing and isolation to enable even more lightweight environments.
Servers also needed to handle more and more network connections and spawning an OS thread per connection became problematic so developers used different programming patterns, async implementations, or user-space threads/processes to tackle this problem. Lunatic solves this by using Erlang’s proven approach [2].
How we develop software has also changed. Today most of our application consists of third-party libraries and it’s common to have hundreds or even thousands of dependencies and obviously it’s impossible to audit them all. When you compile and run your application, the whole code has the same privileges, so a malicious dependency could easily steal your private keys.[3,4,5] WebAssembly is trying to standardise ideas like Interface Types and dynamic linking between Wasm modules. We could isolate libraries into different modules based on capabilities they require (which “system calls” they use) and let developers decide which parts of their app have what capabilities.
Other use-cases that Lunatic and Wasm enable are plugin architecture to run third-party code, sharing code between frontend and backend, polyglot codebases that use Wasm interface types to call each other functions, etc.
Currently Lunatic is just a runtime but ultimately we want it to be more like an operating system for server-side applications. The value we want to give to developers is simpler deployment and management of running apps, better capability-based security model, and seamless integration with third-party tools (logging, monitoring, profiling). Ideally all you need to do is compile your app to WebAssembly and you are ready to go.
We have built two demo apps to showcase Lunatic. Lunatic.run (https://lunatic.solutions/run/) turns any command line application into a web server endpoint. Read HTTP requests from stdin and respond to stdout. The other is Lunatic.chat, a telnet chat server written in Rust using actor-based architecture (https://github.com/lunatic-solutions/chat).
We are super excited to work on these problems and we hope we have managed to convey some of that excitement to you. Please share with us your thoughts and questions. Does our big picture resonate with you? Would you like to use a runtime like Lunatic? Do you have some other use-cases in mind?
[0] https://crates.io/crates/async-wormhole [1] https://webassembly.org/docs/security/ [2] https://www.phoenixframework.org/blog/the-road-to-2-million-... [3] https://news.ycombinator.com/item?id=26087064 [4] https://jordan-wright.com/blog/post/2020-11-12-hunting-for-m... [5] https://snyk.io/blog/yet-another-malicious-package-found-in-...