Cool project! I worked on something similar a while ago to learn about automatic differentiation: https://github.com/hexhowells/onegrad.js

Needs more layers to be really useful though.

I always thought examples using browser extensions would be neat since their built in JS and you only need to download the model once.