XNOR.ai frees AI from the prison of the supercomputer | Hacker News Reader

XNOR.ai frees AI from the prison of the supercomputer

techcrunch.com

84 pointsdwynings25 comments

Thought that Google's TensorFlow already does this and has since shortly after it was released - what am I missing?

Example: https://www.tensorflow.org/mobile/

TensorFlow supports weight quantization, but XNOR seems to be rewriting the trained model graph into a one that only uses binary boolean ops, and probably optimizing the graph along the way.

I think this is their paper http://www.arxiv-sanity.com/1603.05279 and this is the (torch) code https://github.com/allenai/XNOR-Net

Cursory look suggests they quantize both the weights and functions during classification on the smart phone and during the forwards and backwards pass during training. But you still have to use a continuous version of your logic gate to get an error gradient during training, so I'm guessing they still do the training on "supercomputers" (a desktop with a gpu really).

That uses lower precision arithmetic. In this case they are reducing the precision to 1 bit, apparently.

XNOR-Net is not currently part of TensorFlow, although there is a feature request to support it. [0]

The main difference is that XNOR-Net uses 1 bit quantized weights and XNOR+popcount to approximate the dot product of convolution, which can be implemented very efficiently by using 64-bit arithmetic instructions on a CPU to operate over 64 1-bit components in parallel. [1]

TensorFlow's optimization is also using discretized weights, but AFAIK targeted at 8-bit quantized weights. [2]

[0] https://github.com/tensorflow/tensorflow/issues/1592

[1] https://arxiv.org/pdf/1603.05279v4.pdf

[2] https://petewarden.com/2015/05/23/why-are-eight-bits-enough-...

TF is not the nicest thing to work with. It is still very computationally expensive to use on embedded hardware platforms.

The license specifies: The following license governs your NON-COMMERCIAL use of the Software. Commercial use is strictly prohibited.

That's a goodbye for anyone wanting to build anything interesting in the future with it. Such a pity. I couldn't find anywhere any kind of "commercial license".

Why is it that only commercial applications are interesting ?

TL;DR, it is NOT that only commercial is interesting; it is that not commercial allowed is really restrictive.

I do a lot of Open Source, so I know the troubles associated with GPL and the like. This is a lot more extreme than that.

If you were to make a tool based on this a lot of your potential userbase dissapears. Like most of the people.

Why? For using it, while I mostly use Open Source, learning/mastering a tool only to discard it and then learn/master a different tool that does the same for a commercial application is a HUGE waste of time.

If there was a way (2 licenses) to use this as commercial software then it'd be a different story.

PS: I'm the author of legally and I made it for this reason: https://npmjs.com/package/legally

Disgusting appropriation of language to punch up a business headline

It's tech journalism, what did you expect?

recent discussion of running tensorflow/grad. descent on DSP chips: https://news.ycombinator.com/item?id=13379847

This is a good writeup on mobile/low power devices, lower precision arithmetic, compression, sparsity, hogwild type "naive" assumptions etc https://arxiv.org/abs/1612.07625

Uncanny Vision[1] have similar computer vision and deep learning libraries for ARM chipsets.

http://www.uncannyvision.com/

Sending data to a server isn't that "comically clumsy", light speed is very fast. https://superuser.com/questions/419070/transatlantic-ping-fa...

Unless you're talking about privacy, there I agree, I would much rather my phone do voice recognition locally than send it to google servers (where it'll be stored [0] so they can train better models later). That's a bit of a catch-22 though, if people don't need to send you their data to use your model then you won't get data for training your model.

[0] https://myactivity.google.com/myactivity?restrict=vaa

Inference (forward pass) in convnets is really cheap even in floats, there are some awesome hardware like NVidia Jetson and most deep learning framework (i.e. Caffe) support it right out of the box. So we were doing realtime inference in embedded for years now, what am I missing?

You're missing the fact that Jetson is $435, it draws a lot of power, and requires cooling.

So where do I buy those xonr.ai, how much does it cost and when do they ship?

Well, if you look at their website, they have an email there, which you could use to ask them directly.

There are lots of starputs like XNOR.AI for example: http://aipoly.com/ http://scorch.ai/ they have dealing with devices also

Rumelhart was the first to note this, I think, although of course he didn't have mobile phones. Implementation looks spiffy, but not spiffier than Han et al's work

Backprop is expensive. Not sure what the revolutionary aspect is here. Smells a bit like the Theranos of AI.

I am pretty sure this is about optimizing the graphs of trained models. That is, training is not freed from "the prison of supercomputers".

Exactly my point. Feed forward can run on much lower specs just fine. So their whole "prison of supercomputers" analogy is for the wrong side of the equation.

This is for optimizing inference. For consumer devices such as smart cameras, speech recognition on your phone, or robotics that's the part that matters.

They have real speed improvements by using 1-bit quantitzed weights and then running operations using 32-bit or 64-bit bitfields which happen to use very few instructions/cycles on modern CPUS: https://github.com/allenai/XNOR-Net

Isn't this article conflating feed forward applications of trained conv nets with the training of conv nets? The latter is what is so expensive, yet it seems like the demo of object recognition is the former, and not a breakthrough to run on a phone AFAIK.

Reply on news.ycombinator.com