Efficient Recurrent Neural Networks using Structured Matrices in FPGAs | Hacker News Reader

Efficient Recurrent Neural Networks using Structured Matrices in FPGAs

arxiv.org

79 pointsgodelmachine19 comments

The main idea is to take each weight matrix W, divide it into smaller blocks, each of dimension, say, n×n, and make each block a circulant matrix that can be specified by a vector of only n elements.[a]

This reduces W's memory consumption by a factor of n, and makes other gains in computational efficiency possible. Read the paper for details.

However, as far as I can tell, it appears the authors have tested this technique only with one RNN architecture, in one task. It's hard to know whether the technique will hold up well in a broad range of RNN architectures/tasks.

[a] https://en.wikipedia.org/wiki/Circulant_matrix

Speech recognition be the task they implemented this for.

What about image recognition? Too complex? Or doable at 320x240 resolution?

How does this compare to GPU? These FPGAs are high end ones and not cheap for sure.

Regarding GPU, they have only said - "The ESE achieves higher energy efficiency than GPU, but its performance is lower and it cannot operate in real time"

So maybe we can assume the performance is quite fast - as compared to the GPU they used in ESE, and this work can operate in real time?

Table 2 has latencys (measured in microseconds) and a frames-per-second measure too (which places it a ~250k-400k fps). According to them, ESE has a 17k fps "theoretical max", but in reality has a lower performance.

I don't really get the fps measure -- without context, I can't say above which threshold they'd be able to perform real-time operation (this is surely due to my lack of knowledge in this domain).

According to [1], a high-end FPGA can do "AlexNet inference performance: int16 over 2,400 img/s, int8 over 4,500 img/s.". Plug in your favorite nVidia numbers and compare.

[1] http://mipsology.com/zebra.html

Thank you very much for the link!

Hi Insru - how do you think GPUs would fare taking into consideration alain94040 's metrics?

Please look at this pdf: https://www.nvidia.com/content/tegra/embedded-systems/pdf/je... There is a table on page 9 and it claims, that Titan X can make 3216 img/s while TUL-KU115 FPGA board can make only >1000 img/s. This KU115 chip has a project price of ~$2000. Real time capability is nice, but that’s still expensive.

Thanks for this PDF :)

Do you think TitanX has real time capability?

I am FPGA developer and have not much knowledge about GPUs. I just can make a guess, that no one can guarantee execution time on GPU. There are all little details well defined in FPGA design and result is always available after defined delay.

Though wouldn't it be far more interesting to compare the latencies at these rates, than the throughput? Roughly similar price and throughput, but very different energy efficiency and possibly latency could be rather significant for quite a few applications.

Xilinx has prepared some marketing material regarding latency: https://forums.xilinx.com/t5/Xcell-Daily-Blog/Xilinx-reVISIO...

For some reasons the Xilinx link is not opening.

Strange. It’s works here. Try this: https://www.xilinx.com/support/documentation/backgrounders/r...

Got it! Thanks :)

> This KU115 chip has a project price of ~$2000.

That seems odd - just the XCKU115-2FLVB2104E FPGA goes for $9000 on DigiKey.

From the original paper: ESE outperforms Core i7 CPU and Pascal Titan X GPU by factors of 43× and 3× on speed, and it is 40× and 11.5× more energy efficient than the CPU and GPU respectively.

Reply on news.ycombinator.com