Hacker News Reader

Comment by jules | Hacker News Reader

Unfortunately this solves the wrong problem. The bottleneck isn't arithmetic, it's data movement. The number of transistors doing arithmetic is already a very tiny fraction of a modern chip. Reducing that tiny fraction to an even tinier fraction by making arithmetic inaccurate isn't a good trade-off.

View on HN

They seem to be focusing on specific applications where arithmetic might be (or so they hypothesize) a bottleneck. His LinkedIn says, "we recently showed a 6400x improvement in speed/power ratio tracking objects in video for the U.S. Navy".

What have happened the last 20 years is that the total amount of arithmetic that can be done has increased tremendously, while communication has not to the same extent. That means that more problems than before is communication bound (even inside a processor) than before. Since problems range from extremly IO bound to entirely CPU-bound, more problems are bound by communication than before.

A friend of mine that worked with supercomputers back in the 90s used to say that a supercomputer was a device that converted a computational problem to a communication problem. Today this is true for mainstream computing as well. There are still problems that can take advantage of the massive amount of computing available (Some mentioned Convolutional Neural Networks), but many tasks that used to be CPU bound is now effectively IO bound.

There's a new application area where arithmetic is the bottleneck, and accuracy is less important: Convolutional neural networks. I would love to see a convnet chip built using these techniques.

There's a clear and massive movement of data forward for prediction in a CNN, and all that plus saving intermediate state and then moving the same amount of data backwards when performing backpropagation.

But don't believe me, don't believe a word I type. Go prove the above wrong by building a killer AlexNet implementation out of this chip. See slide 9 of this presentation for why I don't think you can do this (too much data transfer):

http://www.slideshare.net/embeddedvision/tradeoffs-in-implem...

Slide 9 makes it clear that, for most layers computation takes more time than data transfers

If you can reduce the size of the computation units you can have one (or several) for each layer, making it possible to hardwire the transfer between layers

What if we take the arithmetic and move it next to the data?

Yes, I'm still waiting for memristors.

Which reminds me of the theoretical/thermodynamic perspective of specifically copying data rather than movement. State reduction. https://en.wikipedia.org/wiki/Reversible_computing

The bottleneck actually is arithmetic. "GPUs have much higher ALU throughput since the GPU chip area is almost entirely ALU"

http://devblogs.nvidia.com/parallelforall/bidmach-machine-le...

Also on the horizon there is 3d chip manufacturing technology(3d-monolithic) ,with extremely large bandwidth between the two different layers of the chip,possibly being gpu + dram.

The bottleneck is often RAM. This is especially clear when writing performance-oriented code in CUDA, where the amount of cores (threads) per one shared memory controller is in the order of thousands.

The bottleneck is not arithmetic for a long time, it's data movement. Arithmetic is practically free nowadays. See presentation by Horst Simon (Deputy Director of Lawrence Berkeley National Laboratory) "No exascale for you!" [0]

The energy cost of transferring a single data word to a distance of 5mm on-chip is higher than the cost of a single FLOP (20 pico-Joules/bit). 5mm =~ the distance to L2 cache or another CPU core. The cost of transferring data off-chip (3D chip and/or RAM) is orders-of-magnitude higher, see graph.

[0] http://iwcse.phys.ntu.edu.tw/plenary/HorstSimon_IWCSE2013.pd...

...but since GPU's already exist, they kind of are the "already at large scale production" solution to the problem. For very little money you can get some pretty insane single precision throughput for SIMD calculations.

What you run in to are problems feeding the beast data fast enough.

Reply on news.ycombinator.com