TensorFlow Lite is for mobile and embedded devices | Hacker News Reader

TensorFlow Lite is for mobile and embedded devices

tensorflow.org

101 pointstosh22 comments

TensorFlow Lite sounds like a good idea, but they don't talk at all about how to build it. "Run this shell script" is neither saying what the requirements are, nor does it give any idea about supported targets. How I hate this new trend of not documenting anything.

This might be usable if it doesn't require Bazil. Requiring that makes deployment much, much heavier and makes the whole process significantly more complex.

"Embedded"? Does that mean ARM to these folks who don't know how to version and how to document?

I'm not a fan of Bazel either. Luckily, this doesn't require Bazel. Their guide [1] explains how to build using regular make for RPi.

[1]: https://www.tensorflow.org/lite/rpi

I'm a big fan of Bazel (or its Google internal counterpart, Blaze). Such a big fan, in fact, that I even use it for Python projects, and package outputs into Google PAR: https://github.com/google/subpar.

As long as you stick with the official 4 Google languages (C++, Java, Python, Go) it doesn't get any better IMO, as far as build systems are concerned. I do wish it didn't require Java though.

I have a bad impression of Bazel for multiple reasons.

I wanted to build TF on the Pi but at the time I assumed it required Bazel. Bazel had no ARM version (my experience was in mid-2017). I had to first build Bazel from sources. Failed a few times because its heap settings were too low, required a lot more swap and storage than I expected, and got the Pi so hot that I had to quickly setup some cooling fans around. A successful build took ~1.5 hours on a RPi 2.

Then the full TF build using Bazel took some 10-11 hours.

Finally, Bazel's command-lines - ex: "bazel build --config=opt -k //tensorflow/tools/pip_package:build_pip_package" - felt very counterintuitive and not easy to remember.

For the first two problems, I learnt it the hard way that a cross-compilation environment is better than building on-device. The last one I still have a problem with.

Bazel makes it reasonably straightforward to set up crosstool (a cross-compilation toolchain). That's how it's supposed to be done, whoever ported TFLite to Pi just probably neglected to provide a crosstool for it. It's a much better dev experience if you have a crosstool. Rebuilds on changes are near instantaneous and fully incremental. Toolchain can be set up to only be downloaded as needed. A good PR idea for someone, if it's not already done?

I managed to build it just fine. Their shell script didn't work for Raspberry Pi at the time, but fixing it was about 5 minutes of work.

The way NDK related deployments work is still a joke in Android, despite some improvements to correct the situation.

Most Google native libraries targeted to Android, instead of using Android tooling integration, tend to be some link to a github repository with an endless list of manual integration steps, including making use of Blaze.

It is as if the teams don't know better, never used Apple or Microsoft's C++ SDKs, or at very least walked next door to check how one is supposed to do it on Android.

Ten years ago I wouldn't have predicted a device for ~$5 that could host a wireless webserver (e.g. ESP8266). But nowadays it doesn't take much of a stretch to imagine that very powerful and cheap general purpose ML devices - kinda like Marshall envisioned in manna - are going to flourish in the next 2-3 years.

They have a section for “Companies using TensorFlow Lite,” but half of them are Google apps.

They say performance is comparable, but I wish the landing page would elaborate more on the differences between this library and standard TF.

shh -

"TensorFlow Lite is the official solution for running machine learning models on mobile and embedded devices."

it's the OFFICIAL SOLUTION.

please, DO NOT QUESTION us. We are TEH GOOGZ!

I have a raspberry pi running a fairly simple application. What can I do something which might be useful using tf lite on my pi?

Image or facial recognition. You could make a home security system.

You just need to run a pretrained model for that right ?

you could use a pretrained object detection model with some lightweight logic built around it (e.g. send you a slack notification when a human object is detected in at least 5 consecutive frames between these hours) but you might also want to find-tune a pretrained model to work well given your camera's angle, lighting conditions, etc.

First Robotics is a worldwide robotics competition that was started years ago by Dean Kamen. This year’s challenge is called Rover Ruckus and simulates a Rover landing on an alien planet. There is a 30 second autonomous mode wherein one of the challenges is to use a camera to discern where a gold mineral lies in a field of two silver and one gold minerals. My son’s team uses Tensor Flow to accurately discern which mineral is the gold.

For reference, this challenge is part of FIRST Tech Challenge (FTC), in which they build robots the size of 18” cubes. FIRST Robotics Competition (FRC), which builds much larger robots (120 lbs, often as tall as people) has its competition announced in two weeks. Many teams that are part of FRC use Raspberry Pis or other microcontrollers to do vision processing and it’s super cool to hear that FTC teams are doing it as well.

Both are very exciting and great opportunities for high schoolers to get involved in STEM (as well as being an amazing chance for adults to get involved in mentoring a team). I can’t recommend it enough.

My intern, Vasu Agarwal, built FTC's official tensorflow detector for this year's competition. I was pretty impressed how quickly we were able to turn a model trained on a GPU into something that ran at reasonable FPS on Androids. We used TF Lite, I loved working with it. Heck, I enjoyed using it more than TF proper.

is there a blog post on this ? we have been struggling to export models to tf-lite. Not sure if you are using TPU trained models (which is cheap) or GPU trained ones using your machines... but a quick pointer would be helpful.

Maybe http://firsttechchallenge.blogspot.com/2018/11/tensorflow-ma... and https://github.com/ftctechnh/ftc_app/wiki/Using-TensorFlow-L... ?

I enjoy how Googlers appear to regularly name drop interns they’ve hosted as if the rest of us should know who they are. Never seen that at any other institution.

I didn’t read it that way. I read it as being very proud of a junior colleague and wanting to give him praise publicly and by name.

I was just name-crediting my intern (who did the hard work) since we didn't publish a paper. Never meant to imply you should know him.

Reply on news.ycombinator.com