erikkaum

I've been interested in ZML for a long time: how does it work, what makes it good? To understand the stack properly, so I built a minimal version of ZML in Rust. And wrote a blog about it. It's essentially a trace-based tensor compiler for Rust. Build computation graphs with a familiar tensor API, lower them to StableHLO MLIR, and execute through PJRT on CPU.

It even runs SmolLM2 on CPU, 5 tok/s.

The idea came when tinkering with WASM+WASI and realizing that you can compile the Python interpreter to wasm32-unknown-wasi, and run it in a WASM sandbox. It's an experimental Rust server that executes Python code. I've always been amazed by the Cloudflare Workers runtime, which builds on the V8 engine. You can think of this as an aspiration to build something:

- with similar startup times (and maybe similar memory footprint) - that's not only for JavaScript - open-source for you to run locally and tinker with

Happy hacking!