Hacker News Reader

Comment by sarreph | Hacker News Reader

From the article:

> Within the last years, multiple Javascript frameworks were developed that can help you to create, train and use Neural Networks for different purposes.

Due to the popular discourse on how 'unsuitable' JS is for anything but front-end development°, does anybody have any insight on how the efficiency of JS neural net libraries compares to that of the classic Python et al. tools? I've been trying to get started in neural nets for a while now, but am curious as to if learning it via JS will be a limiting factor that I will ultimately have to switch to a more efficient/suited platform.

° Something that I do not agree with myself, as a Node.js and back-end JS developer

View on HN

I wrote the the original machine learning for our company using a fork of brain.js. We launched to 300k+ users the first day with a model training time of under 10 seconds. I've since re-written our machine learning in Go. For getting something out the door quickly, Javascript is great.

It worked well for us because we had to stream the data into the network for training, --all the training data did not fit into RAM on Heroku's 1G dyno. Node.js does this IO transfer really well. (Full disclosure, I wrote the streaming implementation for brain.js.)

Also, training artificial neural networks is a serial computation for the most part. (Lots of sigmas). There are parallel optimizations you can make but probably won't see the benefits (in terms of training speed) until you have high dimensional data. Doing this sort of thing on a GPU is also orders of magnitude faster.

One thing to note however, if you plan on serving a high number predictions from trained models in Javascript, you will see blocking because each prediction does require a small bit of computation and Javascript / Node.js is single threaded.

You wouldn't happen to have written about your adventures, would you?

No, I haven't written anything.

JS is mostly 'unsuitable' because of the lack of integration into GPGPU.

This is one of the reasons I was hoping WebCL[1] (now dead[2]) would take off. It would have enabled GPGPU work that is pretty much a prerequisite for anything that would be taken seriously by the community.

There was talk of ARB_compute_shader support as mentioned in the second ref link, but I haven't seen anything else in that area.

[1] https://www.khronos.org/webcl/

[2]https://bugzilla.mozilla.org/show_bug.cgi?id=664147#c30

JS is mostly 'unsuitable' because of the lack of integration into GPGPU.

Does that mean that using an already trained model would be slow? If I wrote something to train a model on a GPU, could I then export the data to use with JS in a browser?

Training is essentially running the net forward then backward on many training examples. Using the model is running it forward on one input, so it would be much faster depending on your specific model and use cases

For any non trivial deep network, any CPU based training is horrendously slow, even if multithreaded. So, Javascript is doomed at it. Python is not, as it can interact with native binaries which may talk with the GPU.

> Something that I do not agree with myself, as a Node.js and back-end JS developer

NodeJS mostly works when CPU time is not the limiting factor, that is, powering the Web. Neural network is the polar opposite.

Well, somebody could write a NodeJS library that talks to the GPU.

Atwood's Law makes it inevitable that someone will if no one has already.

Absolutely true. https://github.com/kashif/node-cuda

Also of note: https://github.com/nikhilk/node-tensorflow

I think they mean that JS is good for more than just FE which doesn't necessarily mean Neural Networks..

> Python is not, as it can interact with native binaries

It sounds like what you actually mean here is that Python is equally as unsuitable and must instead use native binaries. Which one could just as easily do with JS...

A reasonable argument could be that such native binaries are available in the Python community and already in use as stable existing solutions, but that's more about ecosystem than language suitability.

Additionally, while I'm not too sure what convnet, node-mind, brain.js, etc. referenced in the article are using internally, there are gpu-oriented solutions being written in JS like https://www.npmjs.com/package/weblas

Python is dog slow. This is why it is used as a glue language for C modules. This is the canonical way to do things. Javascript OTOH is decently fast on its own, but you can't really interact with libraries written in C or FORTRAN, as far as I know.

https://github.com/node-ffi/node-ffi

Cool, good to know.

Node has been able to interoperate with C/C for some time, via node-gyp and node-ffi.

The current frameworks are deeply (ha!) unsuitable for any serious work, by which I mean anything where a neural network would approach outperform other methods.

However there is some possibility that technologies like WebCL and replacements will overcome this limitation by allowing access to GPU resources. Further reading: [1][2][3]

[1] https://github.com/waylonflinn/weblas

[2] https://github.com/karpathy/convnetjs/issues/13

[3] https://github.com/mikeseven/node-webcl

[4] https://github.com/stormcolor/webclgl

I would guess that it is very unsuitable, since for anything but the most trivial models, even those 'Python' libraries resort to highly-optimized (compiled) C subroutines, especially for the training part. Even then you require a highly nontrivial amount of time to train the network, especially if you are working with medium to large datasets.

Later on, when the speed is not so critical, you can do inference with those pretrained weights and your model implemented in Javascript.

With javascript you have ASM.js which should get you close to native C speed for exactly these types of operations.

But I don't know how well you can talk to the GPU

not as close as you think it is. perhaps C is still 2x, 3x faster.

Most Python neural network libraries actually have the critical parts written in C++, and use GPUs. JavaScript has asm.js, which isn't as fast. There's also NaCl in some browsers.

What JS really needs is WebCL. That will revolutionize JS capabilities.

What we need is support for optimized/hw accelerated versions of JS libraries in browser.

Eample:

A page declares that it will use library foo, e.g. <lib name="foo" version="^1.0"/>. Browser, e.g. firefox, looks into common repository and sees that there is two versions of "foo": one generic, and one optimized for firefox-x86_64, with native code, so it used optimized version instead.

Of course, code must come from trusted source and be signed, to reduce risks.

A corollary of Parkinson's Law is that, whatever language you start with, your neural net project will expand to fill the processing power available to it.

Reply on news.ycombinator.com