Hacker News Reader

Parent Full thread

SparkyMcUnicorn

Yes. MLC's inference engine runs on WebGPU/WASM.

https://github.com/mlc-ai/web-llm-chat

https://github.com/mlc-ai/mlc-llm

https://github.com/mlc-ai/web-llm

View on HN

Yeah, something like that, but without the WebGPU requirement.

Neither FireFox nor Chromium support WebGPU on Linux. Maybe behind flags. But before using a technology, I would wait until it is available in the default config.

Lets see when browsers will bring WebGPU to Linux.

This should be what you're looking for. It doesn't utilize the GPU, but WebGL support is in the TODOs.

https://github.com/ngxson/wllama

https://huggingface.co/spaces/ngxson/wllama

Firefox Nightly on macOS now supports WebGPU, and the documentation says the Linux build supports it too.

Reply on news.ycombinator.com