Neural networks in JavaScript – free 19-part course
scrimba.com
scrimba.com
Only thing is, I kind of wish there were something like the time-coded comments SoundCloud has. At the end of the second guide, we're asked to play around with the tests.
I added: console.log(net.run([0, 4])); console.log(net.run([3, 3])); console.log(net.run([8, 4]));
Based on the training data, I would expect this to resolve to ~4(or 1), ~0, ~8(or 1) by standard logic expectancies (if same return 0, else return the higher number or 1). But instead I received ~0, ~0, ~0.
It's not immediately obvious what is causing this. But it seems like the model created is inherently ignorant of basic logic (at least by my narrow definition), and there isn't any immediate discussion of caveats as to error margin.
I'll admit this might be a n00bish concern based on never programing neural nets before, but as this guide seems focused on introducing NN's to n00bs like me: a way to discuss concerns with other viewers/the author would be amazing.
Aside from that, incredible work! I'll keep watching to see if I can figure out my misunderstandings.
Update: Just discovered the Q&A tab, this should likely be adequate for my concerns. Well done. This may be the best online demo/tutorial I've ever seen.
Couple of thoughts:
- The "hello world" of ML seems to have kinda settled on CNN "cat detector" or MNIST type things now (for better or worse). It does not appear that brain.js has support for CNNs yet(?), but it might be nice to address this in the course somehow - even if it is just "this is coming soon". Lots of people are visual learners - it would be neat for this to be supported somehow, e.g. for brain.js to support loading training data via URLs or grabbing a frame from a webcam and doing some basic stuff with images. This kinda goes for the brain.js webpage too - the demo there is a bit underwhelming compared to the fun tensorflow playground @ https://playground.tensorflow.org/
- It might be nice to see a few more "real world" examples of doing things that real people might want to do, e.g. a comment toxicity/spam detector for their sites that runs real-time in the browser. From this course I am not sure how to load in a pre-trained model for instance.
- I tried to use the GPU NeuralNetwork in a couple of the sessions but I got a "TypeError: array is undefined (1.6.0/browser.js:18548)" error.
I personally haven't seen any NNs being used in browser apps, but there are plenty of existing mobile apps that has NNs to classify audio/video/etc directly on the device.
for most deep learning models, training on the client is completely unreasonable, as they require weeks of training even on multi thousand $ pro equipement.
For training, tiny metalearning models are the only reasonable thing to train on the clients in most useful scenarios, as they are pretrained to take as few examples as possible to train on a specific task (see MAML).
For inference, aside from educational applications, the only potential advantage of in browser over in server is the lower latency. the main disadvantages are that you need to send the model to the client (multiple MB), with the longer loading time and potential problems with intellectual property this entails. maybe for some extremely small models in very latency driven applications, it can be worth it.
So, overall, either educational purposes, training and use of latency hungry tiny metalearning models or inference with tiny pretrained latency hungry models, like computer vision on webcam sort of deal
We're adding GPU support that use either client side OR server side GPU, so that any case you mention can be handled.
The idea here is that the net can continue to train, reinforcing its previous understanding with new understandings, if new training data is provided.
It may be that we need to clarify the tutorial, as your point is based around unsupervised learning, not having training data.
Is it an overloaded term? YES!
One complaint... `#` THIS IS NOT AN ASTERISK! (he said it like 5 times in one video)
Otherwise great tutorial
Anyone know if anyone is tracking this (similar to github language usage stats)?
- `this`
- the whole prototypal inheritance thing
- class inheritance bolted on top of prototypal inheritance
- arrow functions vs `function` functions
I think we're at a point where you'd have to justify why you didn't use Javascript for a learning course.
Some demos such as real time object detection isn't possible at all if you had to pay roundtrip server latency (not to mention the complexity of streaming video to and from your server): https://github.com/ModelDepot/tfjs-yolo-tiny
And a lot of web demos such https://affinelayer.com/pixsrv/ I don't believe will be up for such a long time if the author had to pay ongoing server costs.
OP has a point, if I want to learn PRACTICAL front end programming, I will choose javascript, not Python. Same for Neural networks, just switch the place.
JS is faster than Python, but in the land of DL, there is C++ and everyone else. No one is using Python to do the actual computation anyway.
Depends what "actual" computation is. If you're definition of "actual computation" is something which requires extreme optimization then your definition precludes the question. In the real world there is an extraordinary amount of computation done with inefficient languages though simply because development time costs very often outweighs run time costs.
The point is that none of computations are done in pure Python. Python just provides a convenient wrapper over non-Python code.
To deploy a model in js (as a web page) all you need is a static S3 or GCS bucket. You don't even need a webserver and it can automatically handle infinite scale. Show me a python solution that can do the same.
First of all, what you have described is far from the reality.
Had this come true, only inference will be in javascript, through some language agnostic standardized model format, not training. The model is just a blackbox function for the js runtime to call. The amount of javascript to make this happen will be surprisingly slim anyway.
Did you try any of the links I included? This is the reality for all of them and they are a few years old. They have a model file loaded from bucket url and never make another network request thereafter.
It's actually the recommended workflow from https://js.tensorflow.org/ where you'd find tons of other examples.
But you're right, this is for inference only. I would not do training in JS.
Though if you only meant performing inference in user's browsers, then the challenge would be to find a way for TF/pytorch pre-trained models to port and perform accurately in js. If successfully done, I can see some use cases here.
Doing the same thing in Javascript with good old-fashioned arrays using good old-fashioned terminology clears the fog and makes things simpler for people who are not already fluent in numpy's data structures and terminology
Javascript is - in my view - the equivalent of a "business english" of programming, i.e. even if you aren't fluent, its syntax and terminology is familiar to C/C++/Java/C#/Golang/ObjectiveC/Perl/etc that most people will at least be able to understand what is going on in the same way that business people who might not be fluent in English will at least be able to understand and basically communicate with each other even if they perhaps will not be writing Sonnets. Python feels like a niche language that developed in isolation and is only readable to people who have actually gone out of their way to learn it.