Tensorflow.js – A Practical Guide
blog.yellowant.com
blog.yellowant.com
https://js.tensorflow.org/tutorials/import-keras.html
Training a model in the browser is the least practical use for tensorflow.js IMO (unless maybe you want to hijack people's browsers to help with training or something).
Unless you want to do some type of federated training in real time without sending private information to the servers (very unusual case in my opinion)
HN people are mainly interested in it because it's one of the major frameworks for creating neural networks. Also because Google.
The framework is written in C++ but has decent bindings to Python, and is popular in both languages. Aside from wanting to use Javascript, there are no good technical reasons to do any of this stuff in Javascript.
- Many companies and projects have their entire server-side stack in JavaScript and Node.js, and often they want to simply make a prediction through a model. It's quite a lot to ask them to pull in a python runtime just to make a prediction. TensorFlow.js with node bindings to TensorFlow C enables this type of inference with minimal overhead.
- Privacy. You can make predictions locally, or send embeddings back to a server without the raw data ever leaving a client.
- Flexibility of JavaScript / TypeScript. Dynamic languages are great for scientific computing, TypeScript allows you to define your own level of type safety, from raw JS on one end, to strict typing support on the other end.
- Interactivity / education tooling. See tensorflow playground for an excellent example.
- No servers for applications. Making predictions in TensorFlow on a server can be expensive in the long run. Hosting static weights on a server is much much cheaper.
JavaScript and Python ecosystems for machine learning are not mutually exclusive -- they both have their strengths and weaknesses.
"Many companies and projects have their entire server-side stack in JavaScript and Node.js, and often they want to simply make a prediction through a model."
Right. So this is "we don't want to use another language". Acknowledged.
"Privacy. You can make predictions locally, or send embeddings back to a server without the raw data ever leaving a client."
If calling out to a binary is a security problem for you, you have bigger problems than choice of language. Also, of course, you don't need tensorflow to convert your top-secret data into an input vector that you can send somewhere (seriously: it does not help with this problem).
Your third and fourth points -- flexibility and interactivity -- are indeed why people use Python vs C++ (even though it's more difficult and painful to get decent performance out of TF with that approach). So again, this boils down to "I don't want to use Python and I'd prefer to use JS instead."
"No servers for applications. Making predictions in TensorFlow on a server can be expensive in the long run. Hosting static weights on a server is much much cheaper."
You're contradicting yourself with this point. Servers are expensive so hosting static weights on a server is cheaper? I have no idea what this means.
Yup, which is a perfectly valid reason.
> If calling out to a binary is a security problem for you
You miss the point. The security problem is sending the raw data from the client to the server.
> So again, this boils down to "I don't want to use Python and I'd prefer to use JS instead."
Which once again, is a perfectly valid reason
> Servers are expensive so hosting static weights on a server is cheaper?
No, CPU/GPU intensive tasks on a server is expensive but storing a few static weights is cheap.
So don't do that. If you can run tensorflow on your device, you can call out to a local process.
If you want to use JS to do everything, fine. But that's not a good reason. It's just a reason.
Try to understand what is at play here. It's not all about raw performance, there are many more important things to consider that GP explained extensively.
Not from a webapp (without jumping through a dozen other hoops.) With tensorflow.js, you can do (for example) pose estimation, or face detection, or audio recognition, right in the browser without sending data to a remote server.
> But that's not a good reason. It's just a reason.
Yes, of course it's a reason. The point is that it can be a good reason in many cases.
OK, fine. That's a niche use-case for a domain where scale and performance matter so much that we're building specialized hardware to support it. For 99.99% of developers, they would be better advised to find another way to solve their problem using more conventional tools.
There was a guy who built a life-sized house out of Lego once. It was a cool trick, but the difference between him and "modern Javascript" developers is that he didn't try to make anyone live in the house.
Also he isn't contradicting himself, serving static data off a CDN or server is a lot easier and a lot less expensive than having a cluster setup to do inference.
For example, x can be tens of thousands of images of cats and dogs, and y can be 1 for a dog and 0 for a cat. The goal for the fitting algorithm is to find parameters that describe the concept of a cat and a dog so that it can can generalize and categorize general images of cats and dogs. A bad fit would be if the network just memorized the example images.
- Tensorflow is the best tool to make the grunt work necessary to calibrate the parameters of fitting algorithms.
- Models of fitting algorithms are human proposed and where the ML 'art' is. Seems to be kind of reverse engineering.
It is one of many competing tools, which is “best” depends ironically on many variables. CNTK is a rival for example.
What people want is an easy way to express these functions so that a computer can optimize it for them. Tensorflow allows people to do just that, it lets you represent your mathematical equation in a way that can be analyzed and optimized by a computer.
It does it by exposing various mathematical operations that it understands[1]. As long as you can express your mathematical function using these operations then Tensorflow knows how to compute it efficiently and you can use its optimizer to find the optimal inputs.
comes in handy for a lot of problems.
Guide here:
Look into quantizing the weights (e.g., reduce from 32bit per parameter to 8 bits) to reduce model size: https://www.tensorflow.org/mobile/optimizing#model_size
I think your step 4 code is messed up though?
Joking aside this is super cool