Hi HN, I've been obsessed with the idea of visual ML model building since mid-2024. I experimented with a bunch of approaches: Three.js to PyTorch, a Three.js to OCaml compiler with a custom Lisp-inspired IR targeting CUDA. None of them felt right, so I paused to work on other projects.
In December I rethought the whole approach. Instead of targeting CUDA (which requires a server), I asked: what if the entire compiler ran in the browser? WebGPU had matured enough that it seemed possible.
So I built this. ReScript compiles to JavaScript and generates WGSL compute shaders. I defined 208 operations as algebraic types, wrote a shape inference engine with broadcasting and convolution rules, and built a codegen module (3500+ lines) that emits optimized WGSL for every op. The compiler handles topological sorting, buffer allocation, and dispatch scheduling. Autograd generates backward kernels for each differentiable operation. No server, no ONNX runtime—just your browser and GPU.
The demo is a 3D block builder where you drag layers onto a canvas and watch four code views update live: the high-level nn.js API, the ReScript compiler graph, the raw WGSL shaders, and the dispatch table. You can load preset architectures (CNN, Transformer, Autoencoder, LSTM) or build your own.
What I'm most pleased with is the INT4 quantized matmul—column-major weight layout for coalesced reads, vec4 dot products, per-group dequantization. I got Qwen 2.5 7B running entirely in-browser at decent speed even on my crappy used ThinkPad.
Source:
https://github.com/graphpilled/visual-web-ai
Try it out and tell me what you think.