Brain.js: GPU Accelerated Neural Networks in JavaScript | Hacker News Reader

Brain.js: GPU Accelerated Neural Networks in JavaScript

brain.js.org

222 pointsandsoitis57 comments

Long time ago wanted to so something good and took a part in W3C webmachinelearning standardization group. Inspired by simplicity of brain.js wanted browser API to offer something similar to it - simple and easy to learn and easy to understand, also wanted a training API to be available. It is possible to keep simplicity of brain.js and flexibility of any other op set. And my oh my - did hit the wall HARD! Paid seats in W3C rules. They do what they want there. Your voice there matters only if it aligns well with their voice. Browsers API is now their own domain and whenever there is a conflict of interest their choice is obvious.

That group has been around for so long, seems mostly Intel folks.

I just don’t get why the progress is so slow!

I was trying to follow along, and joined calls, but the group moves like a glacier.

If you don't get buy-in from the actual browser vendors, they can just ignore your W3C standards. Buy a couple drinks for the Chrome engineering and product managers and your problem will be solved.

Webmachinelearning sounds like a problem nobody has. Why would I want a standardized browser API for doing stuff that I can obviously already do without such an API?

I ABSOLUTELY loved this project and used it for all of my "quick and dirty" ML stuff where I was fooling around... the only problem, which I don't think has been fixed is that it still requires *Python 2.7*.

End of life for Python 2.7 was 2020!

Yeah this is a big issue that is super annoying to fix on latest Ubuntu.

That said if you can get it working the api is very simple but you can do a lot of things with it.

I created a LSTM network to predict your next move in rocker, paper, scissors in very little code.

What do you mean? Brain.js is JavaScript ML engine.

Check out the README: https://github.com/BrainJS/brain.js

Oh, you don't need to use python if you don't want to. Brain.js happily runs in a broswer.

It's just a requirement. Python 2 is available on all the systems I know. What difficulty are you facing in installing it?

> Python 2 is unsupported by the Python Foundation since 2020-01-01 and is being removed as of Debian 11 (Bullseye)

https://wiki.debian.org/Python

It's not being removed. It won't be default, but installing python2 is possible(https://packages.debian.org/stable/python/python2)

It sounds like the problem is with its dependency with headless-gl, but still you have a point

As others say, it's headless-gl, and that in turn seems to be due to node-gyp.

However the good news seems this is probably more a case of documentation inertia than actual requirements: it looks like you actually can install a version of node-gyp that works with Python 3.x and then in turn should be able to get headless-gl working too, which then let's brain.js work with a non -EOL version of Python

Thanks for pointing this out. I've submitted a PR to resolve this: https://github.com/gpujs/gpu.js/issues/757

That being said, if you're not building from source (you're running an LTS version of node on a supported platform), you don't need to worry about python or many of the build deps.

Correct PR link: https://github.com/gpujs/gpu.js/pull/768

I appreciate your actionism. This is one of the reasons I love HN - super tech savvy people helping each other out. Kudos.

See also https://github.com/webonnx/wonnx, which runs ONNX models through WebGPU. It is written in Rust and can be used natively on Windows, macOS and Linux (through wgpu which uses either DX12, Vulkan or Metal) or on the web (it has a JS API that uses WASM under the hood, which calls into the browser’s WebGPU API). There is a Python module as well.

I've ran across this before, but haven't figured out if a GPU is required to use it. That would be a deal-breaker for any consumer facing web app right?

It will work with software implementations (e.g. llvmpipe/lavapipe is used in CI). That will be slower though. However you’d be surprised at how many devices have a usable GPU. Not all have NVIDIA RTX’es of course, but a simple iGPU will do. I tested it on a fairly low specced Dell office PC with an Intel Iris iGPU and that works just fine. Also most mobile devices will have a GPU these days.

Thanks! it looks like the wonnx CLI itself falls back to tract to do inference on CPU if a GPU is not available[0], and it also sounds like setting up llvmpipe/lavapipe on WASM is much harder (if not impossible?) than just shipping tract, so the approach I'll take is probably a wonnx+tract approach.

[0] https://github.com/webonnx/wonnx/issues/116#issuecomment-114...

I'm curious if there are many scenarios where a CPU fallback, especially javascript based, would have acceptable performance when a GPU is not present. Is that even really a solution?

I'm not far enough into my project to know for sure, but I think that, for example, using tract to do speech recognition can be done in sub-realtime (ie, the inference takes shorter than the audio duration) on a Macbook M1 Pro. Hopefully on less powerful devices too, but I haven't tested that yet.

There's also TensorFlow.js: https://www.tensorflow.org/js

also ONNX.js, which is great for interop: https://github.com/microsoft/onnxjs

https://dannjs.org/

Does it support GPU?

I used their facial landmark model, took some messing around (import/dep issues) but worked in my react project.

My use case was just automatic picture taking when a face was detected in a video stream.

brain.js is significantly simpler to deploy basic projects than TensorFlow.js

For object detection, we made a way to do it from scratch in a couple of hours (for free if your project is open source[1]): https://docs.roboflow.com/inference/web-browser

[1] https://blog.roboflow.com/computer-vision-datasets-and-apis/

I was checking it out, but it seems limited in terms of the type of models you can implement. You don't have CNNs for example.

CNNs are possible, but require some hacky code. Tensorflow.js is far better in this regard.

There's also tract, from Sonos[0]. 100% Rust[1]. I'm currently using it to do speech recognition with a variant of the Conformer architecture (exported to ONNX).

The final goal is to do it in WASM client-side.

[0] https://github.com/sonos/tract

[1] To be clear, I'm not saying that writing in Rust is some inherent advantage for an NN inference engine. But I personally gravitate to using libraries written in languages I know/want to learn, and Rust might be that for some people.

For inference, models should be compiled down directly to WASM or WebGPU or whatever, right? The driving language really shouldn't matter at the end of the day.

Unless you've got massive compute bound transformers or old-school full convolutions, if you're interpreting a list of operations you're going to lose perf.

> For inference, models should be compiled down directly to WASM or WebGPU

I mean, yes? Whether for inference or for training, ML operators are usually implemented in compiled low level languages. That implementation might also take advantage of GPUs.

> ... if you're going interpreting a list of operations ...

I'm not sure what "interpreting a list of operators" means here.

To clarify why I mentioned the implementation language, I recently had to modify the source code of Tract to get my model running (I will push that change upstream at some point). I would not have (been able to do that), had tract been written in C++, for example.

So being semi-retired (unable to find work due to 10yr+ past background issues) and living in Vegas, I've really given serious thought to building some sort of NN to help with gambling in general and specifically the picking of horses (or perhaps even baseball outcomes).

I can't help but think that buried in all the race data their are correlations that are impossible to figure with standard statistical methods but can be sussed out using NNs. Of course I am vaguely aware that many people are already using NNs to predict race outcomes, but I am purposely trying to keep that info at arms length as I don't really want to be negatively influenced existing attitudes.

However, if someone does happen to know of a good NN/horseracing blog that has a positive outlook and is making money, I would be interested in such a site.

Anyway, this type of easy-to-use NN could be exactly what I'm looking for with getting started on this project, and once again, I would be interested in hearing about the pros and cons of using such a lightweight easy-to-use NN for my explorations.

Those standard statical methods probably don't have the gift of thousands of parameters.

Well no, but what’s your point?

It's a pretty strange comparison if you think about it computationally.

I'm very much on the deep learning train but comparisons with traditional statistics are rather somewhat tortured.

I’m still learning about a lot of this stuff so apologies if this is wrong.

Isn’t part of the motivation for using deep learning that it can actually generalize over that many parameters? My understanding is that classical statistical models are prone to overfitting on data as you add more parameters to the model. Deep learning tries to learn the underlying distribution of data, enabling predictions not found in the dataset that still match the underlying positional statistics of the dataset.

In that context I’d say the comparison is apt. It’s not just throwing more compute at the problem - it’s doing so in a way that actually achieves scalable accuracy gains. With pre training and self-supervised learning, you can (feasibly) train such a model without even needing to understand statistics.

There is a good book that I recommend

Precision Statistical and Mathematical Methods in Horse Racing

the book will set you in the right path at least and I learnt some good mental models from it

I just want to clarify I never gambled in my life. But I did have similar thoughts to yours and the book scratched that itch for me.

Tangentially, I'm just curious, is it something that appears in your criminal background check that causes you to not receive offers? Or is it a lack of interviews due to gaps on a resume?

I am a firm believer that if someone demonstrates they have motivation and skill in a writing sample, they deserve an interview. If that person then passes a technical interview, they deserve a job. And if that person is friendly and does not commit any crimes while working, they deserve to keep that job.

    unable to find work due to 10yr+ past background issues

The broken url in your profile might also be a factor.

I had that url active for over 5 years here and never once was contacted by anyone interested in talking to me about a project, so forgive my cynicism with not being very concerned about keeping it alive.

Fpr the sake of not being a total jerk about it, however, I will take it down soon....thanks for reminding me btw!

There might not be enough data to train a NN, and it's probably prone to overfitting the model. It's a similar problem to using NN to predict the stock market. You might have better luck with shallow learning statistical techniques that require less data to train a model.

what do you think the bookmakers are using to set the odds?

If you don't have a PHD level and a large amount of money to start with, don't even try to think about it.

I've spent a few months tinkering with this kind of stuff, with a huuuge dataset, my simulations never went past the annual return of an average ETF, and by eliminating 99/99.5% of bets possible, so you'd only have a few bets that are worth it per month, a bad streak would ruin your return for the year, and you'd have to have access to a lot of bookmakers to get the best odds.

It's just not worth it IMO.

If you can consistently make money the bookies / betting platforms will just ban you.