Thanks for the comment. Parity with llama.cpp isn't my goal.

What I actually want is MoE on machines that can't fit the model in VRAM, and specifically expert-level residency instead of layer offload: track which experts get hit during decode, keep those resident, evict the rest. Doing that well needs the router, the KV cache and the memory manager to be designed together, which is about the only good reason to write a runtime from scratch.

Yes, let’s see! You are welcome to contribute if you like!