How to Create a Neural Network in JavaScript [video] | Hacker News Reader

How to Create a Neural Network in JavaScript [video]

scrimba.com

180 pointsingve42 comments

Having almost failed my last year of IT engineering because of the AI part (which included neural networks) I would have loved working on such technologies, because my teacher only gave us a scientific article about a neural network algorithm and asked us to implement it with Netlogo, which was a huge pain in the a*s to work with. Whereas with this library I understand the code and therefore the logic behind it almost instictively. So to the people behind it : good work, and thanks !

I dislike the way some courses insist on using crusty old academic languages to teach subjects that are much better served by modern real world languages. Is it so students are less likely to copy work from the web? Clunky, poorly documented systems putting people off programming and obfuscating the subject being taught seems ridiculous!

This gave me cold chills about all too many academic projects which led me to question my understanding. Much older and wiser I realize it was the horrid projects that were largely to blame for imposing such horrid implementations as 'learning material'.

You can actually get some useful NN performance if you do this sort of thing in the browser but using shaders in WebGL for doing the calculations on the GPU. There is a Keras JS if you have already trained a net.. I think it uses the GPU. https://transcranial.github.io/keras-js/

I thought the only output you could get from WebGL was pixels... I wonder if they're rendering the output layer on an off-screen canvas and reading the pixel values.

https://developer.mozilla.org/en-US/docs/Web/API/WebGLRender...

Here is another WebGL example that does GPU accelerated handwritten digit recognition. https://github.com/Erkaman/regl-cnn

You have more chances of learning if you understand the math behind.

But if you still are after some code here is mine: http://htmlpreview.github.io/?https://github.com/aguaviva/Ar...

It would be great to add some context to the demo.

I agree. A solid understanding of why it works is still a requirement if you want to do something productive with neural networks.

But as someone who has been taking video courses and seen youtube videos, somehow this was the final step I needed to get started on my idea. It was easy to follow along, the code was nice and clean (most importantly without convoluted variable naming), and the immediately-interactive code made it quick to test a few things, rather than having to fork a project, NPM install a thousand dependencies and then actually do some coding.

Where does one go to learn the math behind it?

That depends if you know calculus, preferably multivariable.

Here is a simple start! https://www.youtube.com/watch?v=N4gDikiec8E

Module one of Andre Karpathy's course is the most intuitive explanation I've seen. Working through backprop equations was a major hassle until I read those notes.

http://cs231n.github.io/

Hey all, Scrimba is on Product Hunt today, if anybody wants to join the discussion there as well. You'll find us on this list:

https://www.producthunt.com/topics/developer-tools

Ha - I chuckled when I saw the [video] tag. I remember when Scrimba was first posted here and it is indeed great -- but the really fun part is the fact that they aren't really videos!

It's painful for us to see that... :p

Hey all, I'm the creator of this screencast (and a co-founder of Scrimba).

Happy to answer any questions people might have :)

Scrimba is a really neat idea. Would you mind going over its architecture and your plan for future language support?

Sure!

The basic idea behind Scrimba is that we record events instead of pixels. So what you're seeing when you're watching a Scrimba screencast is simply a re-creation of what the creator did when creating the screencast (which is done in-browser).

Even the interactions with the live preview are recorder, through a DOM-recorder we've built.

This opens up for tons of possibilities, including enabling the viewer to interact with the code.

We will add support for server side languages (e.g. Python, Ruby, Node) in the future. However, it's not at the top of our list, as there are several core things we need to fix. Once we have a timeline for that, it'll surely be shared on our Gitter channel:

https://gitter.im/scrimba_community/Lobby?utm_source=share-l...

Next up on our feature list is to make the creation process a lot easier, by enabling the creator to edit the recording after it's done. Will likely be launched next week :)

This is a super cool idea really well executed. I wish this existed when I was learning to program.

No code shows up on mobile.

What kind of mobile/browser are you using?

Mixed Content: The page at 'https://scrimba.com/casts/cast-1980' was loaded over HTTPS, but requested an insecure image 'http://i.imgur.com/k4L44Qg.png'. This content should also be served over HTTPS.

switch your imgur to https to fix the SSL for your streamcast. looks really cool though!

Oops, thanks for notifying me about that!

Would really like to find something similar but that teaches how to implement the layers and the network too. I feel like I'm not going to become familiar with NN's until I understand how it's built underneath too.

Does anybody have any good resources for that?

http://neuralnetworksanddeeplearning.com/ also http://playground.tensorflow.org

I've always found Daniel Shiffman's writing to be very approachable (http://natureofcode.com/book/chapter-10-neural-networks/)

You could try looking at Tariq Rashid's video and book: https://www.youtube.com/watch?v=2sevic5Vy4E

https://www.amazon.com/Make-Your-Own-Neural-Network-ebook/dp...

NN as a concept is hard since it is taught with math, however, I found his book to be easier than any other books I've read as it uses minimal math and down to earth explanations. Since NNs can implemented in most languages, I wish that a universal NN language could be developed that anyone could use vs having to learn it from other languages.

That book is only $4 on Kindle. Just purchased it--quite good for a beginner.

It would be great with a tutorial covering something besides the easiest example in the library readme.

It works great if you train it with all possible combinations, but if you leave something out, my experience is it will make very poor predictions.

20,000 rounds for the NN to learn XOR?

Is this typical for this level of task?

By tweaking the learning rate I managed to lower the iterations/rounds to 750.

For my own test project I only did 10k iterations/.3 learning rate on a 16 input, 32 hidden, and 1 output neuron network and my early tests on my "non-math" problem seemed good.

How about creating a fork and explaining what you did to get the iterations down to 750? :)

Interesting, thanks. That's a pretty good decrease in rounds.

Next up: Why create a neural network in JavaScript?

I think with JS being one of the most popular languages around, this can always be answered with 'to enable millions of devs to use this without the need to learn yet another language'

Because you can put your demo online plus your browser is also a nice debugger and profiler. And you can use HTML5 for graphics/fancy stuff that works everywhere.

Because it is one of the most popular and accessible programming languages. It can act as a gateway language for the millions of devs out there to try out NN's without the barrier of learning another language.

Here are some other neural networks in JS:

https://github.com/harthur/brain: Neural Networks in JS

https://github.com/stevenmiller888/mind: a flexible neural network library

https://github.com/cazala/synaptic: an architecture-free neural network library for node.js and the browser

Sllightly off topic, but the Machine Learning department of the University of Tokyo has built a whole set of libraries for machine learning in Javascript: http://mil-tokyo.github.io/miljs.html

<vanity>That moment when you realize that your JS implementation supports conv nets and others don't :D </vanity>

So much value. Cross-platform compatibility (browser/JS, server/Node, mobile/React-Native, robotics/Johny-Five, etc). In-built asynchronous execution of nodes (a boon in ANN architectures).

Then there's dev mindshare. So many people know JS, empowering them would add bodies to meet rising ML demand. I learned Python specifically for TensorFlow. Python's easy to learn, but like any language takes much time to master. I've mastered JS, so Python was a frustrating little reset.

All that said, this cazala/synaptic project doesn't look promising to me save as showcase. Better to focus on exposing JS APIs on existing computation-graph GPU-runnable frameworks, eg node-tensorflow (https://github.com/node-tensorflow/node-tensorflow).

A better question is, "Why not?"

Is there something you'd like to share with us, Johnanneskanybal?

Reply on news.ycombinator.com