Hacker News Reader

You can embed much bigger models into web apps with wasm and WebGL or WebGPU. I have a web app running a 0.6B embedding model client-side.

View on HN

WebGPU is a non-starter for production usage since the support is too limited (No Firefox support, no Linux Support, no Apple x86 support, no MacOS <26).

But yes we are also considering bigger models, though we'll pick the smallest model of sufficient quality because not having to download a 600MB bag of weight is a feature in itself.

That very much depends on your target audience (but also Firefox has partial support these days). However WebGL is widely available and can be used as a fallback - it might be less efficient but for models this small it can still be acceptable.

Reply on news.ycombinator.com