Ask HN: What's the idea behind the WebNN API? | Hacker News Reader

Ask HN: What's the idea behind the WebNN API?

20 pointsjfoster13 comments

The WebNN API will make it possible to run AI models in the browser: https://www.w3.org/TR/webnn/

The idea of having AI in browsers (see the "Application Use Cases") seems cool, but on the other hand, the idea of downloading & storing large models for each website/webapp seems terrible.

It seems to me that it would make more sense to have models built into browsers, and to have a web API that provides standardized ways to interact with the underlying models of each browser. From what I have understood from the WebNN spec, though, despite listing out all of the cool use-cases, the API doesn't seem intended to provide higher level functionality than loading, running & performing operations on a model.

Is my understanding of the API right, or is there something else that uses it to make these use-cases available in browsers?

View on HN

Not all neural network models are necessarily very large, or language models:

For example, many chess engines these days are already based on neural networks, and these run pretty well in WASM (for example, Stockfish, one of the strongest engines, has a 7M and a 44M version on lichess.org), but as far as I understand that's mostly because they're optimized for CPU usage.

It would be pretty cool to be able to run similar specialized models on the best execution resources a device has available and not just the CPU (or maybe a GPU via WebGPU).

That's true for chess engines, but almost every use-case listed in the WebNN specification currently requires a fairly large model. Certainly not things you would want every website downloading duplicate copies of.

> almost every use-case listed in the WebNN specification currently requires a fairly large model

Do they? At least some of them seem much smaller than Stockfish's NNUE.

Could be I suppose, but what about others? I guess the problem I see is that any given webapp might need to download & store several gigabytes.

If the models were built into the browser, the browser would need to download & store the model data, but that can at least be a lot more efficient than websites doubling up on download & storage of models.

A great question, went into a small rabbit hole, still don't know how to answer this xD

MS and Apple already have/will have/build ai features into the OS and intel is also adding AI cores to laptop CPUs.

So when you have a webbrowser based photo editor or other tools, you can build things which do not require a backend server.

Not everything has to run remote and without such an API you can't use it locally.

Perhaps this will allow me/us to write a website for a local agent as well or an assitant.

The question is more boring than that. I agree that running models in the browser would be cool.

The question is, where are the models coming from? If they're downloaded & stored for each website, that's going to get out of hand very quickly. It seems to make sense for browsers to ship with some useful default models, but that doesn't seem to be what the specification is for.

They could run in your local company infrastructure or on my own desktop pc.

I would be happily running my custom agent models on my gpu without sharing those models and still being able to use the browser as an interface.

expect browse/os to ship models and website llms will be just like font/colors in the 90s.

web-safe-models heh.

That's pretty much what I would have expected. I think it makes the most sense for users, since you wouldn't really want to download & store large models for each website. This is not what is covered in the WebNN specification though, and I've yet to hear of any effort to include models in browsers.

yeah, but then it os basically: open on siri/bixby/bard depending on your os.... so, same as doing nothing and sending you to the local app

https://fleetwood.dev/posts/a-case-for-client-side-machine-l...

The question is not whether we need NNs running in the browser, we do, the question is whether WebGPU would be sufficient to cover those needs.

Reply on news.ycombinator.com