It looks like this issue has been resolved to a reasonable extent.

I just tried building TFLite for a lark on a Raspberry Pi 2 and succeeded without much hassle and any of that bazel nonsense. It's a standalone component - they provide a Makefile and I just had to run make.

The example label_image inference app took a bit more work - I had to write my own Makefile - but wasn't difficult. I'm able to run inference on images on RPi2 with both floating and quantized MobileNets tflite models. Will try videos now.

For anybody else interested in trying out TFLite on your desktop or Raspberry Pi or any SBC, I've put up instructions and Makefile at https://pastebin.com/LDEGyG46