Machine Learning in JavaScript
burakkanber.com
burakkanber.com
Let me know if you have any questions. I do intend to keep up with this series, although my pace is pretty slow at about one article every three months or so.
There are already a couple of comments about running ML in JS and how JS and the browser environment isn't terribly suited for heavy calculations. First: you're totally correct; second, I chose JS because it's
1) accessible -- whether you're Python or Ruby or PHP on the backend, you're probably comfortable with JS and
2) it demystifies machine learning -- you have to write your ML from scratch, without the help of all those wonderful Python libs, and I think this exercise shows you that it's not so mysterious after all.
Anyway, thanks for reading, and I'll poke in here throughout the day if you have questions.
I've been building a data management platform for the last 8 years and we are now at the stage where we want to provide tools to help our customers get more from their data than just statistics. As my programming experience is mainly in PHP and JS this set of articles is helping me grasp ML rather than trying to wrap my head around a new language. I'm currently working on k-means clustering and re-implementing everything in PHP to get the best possible understanding I can .. my aim after that is to see how well I can implement things at an SQL level.
>all those wonderful Python libs
As a non-mathematician I have no understanding of how wonderful they really are, which is why this sort of thing is so valuable.
JS and PHP are slow, crappy and bug prone. Sane languages (like Python, C++) have tools to make your job easier (like numpy, blas, eigen and other libraries). They provide fast and reliable math routines so you don't have to worry about some eigenvalue decomposition, matrix multiplication and other problems.
I've even explicitly mentioned that I'm staying away from algorithms that rely on linear algebra, because I'm trying to bring these concepts to people who may not have a CS or mathematical background.
I also agree that other languages offer better tools. For instance, Python has Numpy. However, that's written in C++, not in Python. You can write plugins for Node in C++ too, so nothing would stop someone from writing a Numpy equivalent for JS. You might even be able to run it in some browsers through something like Emscripten with a performance overhead of 2-3x (I think?)
http://cs.stanford.edu/people/karpathy/convnetjs/
http://cs.stanford.edu/people/karpathy/svmjs/demo/
Heather Arthur (npm libraries brain, classifier) has also done a bunch of cool stuff!
For starters, node.js, which makes most of the arguments regarding server/client moot.
Secondly, there are many client side applications for these types of algorithms as well. K-means clustering, for example, is already used by many mapping libraries to group together large numbers of points[1].
I personally use neural networks and affinity propagation in many of my applications for predictive analysis. This does not have to only be educational, or of a 'toy' nature.
[1] http://danzel.github.io/Leaflet.markercluster/example/marker...
Node is a general purpose language that can be used for all kinds of things. I switched from Python to node.js about a year ago for exactly the sort of tasks you are describing and could not be happier. Right off the bat I had huge speed improvements.
Also, io is one of the biggest issues with web based data analysis, so it really should not be underestimated. I can do more with less with node than I could with Python. This is especially true with long running tasks where a 1 minute processing time vs a 20 minute processing time might mean you need 1/20th the number of servers in a cluster ($$$).
Of course, this could be a pretty good argument for something even faster/lower level, but for me, node.js struck a good balance between performance and ease of development/ecosystem. As usual, YMMV.
One last point. The language you choose cannot always be the best language for every task you need. Typically you choose a stack based on the most common/important tasks in your infrastructure, then for less common tasks you just make it work with what the chosen language provides. In this case node.js does not need to be the best solution for ML, it just needs to check the box for being possible, so that devs who needed node.js for other reasons now have the ability to add ML to their toolbox.
Node is a library that can be used for all kinds of things. FTFY
Do hope this author writes more again it has been quite.
For some reason it's somewhat hard to find C-style science code examples in some disciplines. Python feels a bit like a plague in this respect. Everytime I have to wrap my head around while converting code to C-like language (C, C++, PHP, JS).
You need something like numpy to make working in javasctipt easier before there will be a proliferation of of ML in JS.
I really love JS for its distribution and some of the visualizations are amazing. But the low level, numerically stable, matrix math primitives are sorely lacking.
I can see JS useful for a UI to a robot, but I can't see it replacing Python for math, or C++ for speed, or LISP for planning systems.
That said, I can imagine node.js being a better async message router than the current C++ one.
ROS is glued together with XML-RPC, which I think was a mistake (why not JSON???)
> … well, most of the time. There are some things you really can’t do in PHP or Javascript, but those are the more advanced algorithms that require heavy matrix math.
Leaving out javascript (in the browser), it sounds like an odd statement to make about php -- after all one of php strengths is how easy it is to link with c-libraries (or other with c ffi)? Among other things I quickly found:
I would still stick to python. Or java. Or anything else which has a clear syntax and can run at a useful speed (I'm not mentioning C++ because of the coding overhead and dirty tricks which makes it a bit unfriendly for learning an algorithm)
Implying that JavaScript can't "run at a useful speed" is wrong, using modern implementations. This is especially true for code that runs through lots of repetition as the just-in-time compilers in the JS engines do a remarkable job.
Not to mention that viewing JS as a UI-oriented language seems a bit out of date given the 40k or so packages for Node.js that are in npm.
JavaScript of today is pretty different than JS of 2007, and there are more changes coming with generators, iterators, destructuring, class syntax, arrow functions, promises, etc.
I'm actually adding multi-threading to classifier training in node-natural as we speak [0] so it's something I'm recently familiar with. Multi-threading in JS isn't new or particularly exciting (even less so is multithreading in ML/NLP applications) but the marriage of the two has led to a few interesting problems in JS's asynchronous/event based view of the world!
[0]: https://github.com/NaturalNode/natural/issues/124
--
Edited for clarity
But Javascript engines like V8 with its JIT are way faster than Python. You can even use typed arrays that give you almost native speed for such operations (e.g. matrix). I am coding a 3D game in WebGL and JS is as fast as Java when used in a modern fashion, though JS run in every browser
if all you have is a hammer, everything looks like a nail
I think it is a noble thing to explain this in JS. But i don't think "because every body uses js" is a good reason to choose js.
However your specific use case makes sense. But in a broader sense I see more and more people fleeing to JS because its what they know.
I guess fleeing implies that they were using other tools already, but a lot of new devs are going to JS because it just makes sense to start there (lots of flexibility, hyperactive community, education value).
When I took it in university it was taught in language agnostic psuedocode and we were free to use any language from a long list for our assignments.
This feels like begging the question. Why does that need to be the case? Why can't someone strive to learn machine learning _without_ learning a new language? Why can't they get a head start on the concepts early in their career? Is there some requirement that ML _must_ be an advanced topic, only accessible to polyglots that I haven't heard about?
I'm not saying there isn't room for the easier to understand and easier to read guides to ML. More the better, Mitchell was a beast to read through. Its just the language isn't the hard part of the subject. You are the author of the link, correct? I read through some of it, and its approaches the theory and subject matter in a gentle way which is what matters. The sample code is easy to read. I've written maybe 100 lines of js in my life and avoid all web dev like the plague. Your guide is well written and useful. I am not dogging it at all and please don't take it that way. I think its great!
What I'm saying is if someone is saying to themselves "I would be able to learn machine learning if only their was a guide in X" then they are probably mistaken. The code is easy, the math and theory is what is hard.
For you, sure -- but not for everyone.
This series has actually been up for a little over a year now. I get emails from people who didn't know what machine learning was before they started reading the articles, and now they're building some of the most creative and beautiful projects out there. I also get emails from people who need to implement ML in JS or C-like languages but have had trouble seeing the algorithms in full relief when translating from Python, for instance.
The point is, your experience is not everyone's experience. My goal is purely one of accessibility of education. There are smart, talented people who never played with ML simply because they didn't want to dive into a different language, different platform, and different environment just to muck around. There are people who hadn't heard of ML before, but tried it out because JS was right there for them. There are people who stayed away from ML because they thought higher math and a CS education were requirements. Those are facts. This series serves all those people, and it serves them well.
I learned calculus and linear algebra long before learning to code.
He asked "Is there some requirement that ML _must_ be an advanced topic" and I listed a few prerequisite pieces of knowledge that make it fairly advanced. You may learn those prerequisites in a different order but they are required before you properly tackle machine learning without cargoculting through it.
There's money to be made with this combination. The field is ripe.
Good write up too.