A General Neural Network Hardware Architecture on FPGA [pdf] | Hacker News Reader

A General Neural Network Hardware Architecture on FPGA [pdf]

arxiv.org

42 pointsKatydid9 comments

This paper was extremely devoid of content beyond a basic summary of what neural nets are, what an FPGA is, some basics about both (forward propagation, back propagation, usage of HDL modules, usage of LUTs), and a statement that the author created a thing.

As a machine learning guy and total closet FPGA geek, this was sort of a disappointment. I would have liked to see topics addressed like floating point precision, actual benchmarks, how the FPGA can pipeline things better than best known CPU or GPU algorithms due to a lack of pipeline stalls, issues with I/O of training data and predictions, and probably a discussion on LSTM gates or GRUs (which I think the FPGA is particularly suitable for).

Do you even want to use floats when talking about FPGA NNs?

You can talk directly about circuits on bitvectors, of which a subset looks like floats doing math, but your network might have a better encoding than that.

Yup exactly. There is some research on BNN (binary neural networks) but I was curious how this paper approached it (not really anything of substance here)

Tangent, but I suspect you've researched this topic more than I have -- do you know of any good source on the subject of FPGA NNs?

It dawns on me that FPGAs might actually be better than GPUs for a project I've been thinking about, but I know relatively little about state-of-the-art design/synthesis techniques.

I've searched high and low and there isn't much, partially because the barrier to entry to both is really high. I've been experimenting a fair bit and might start blogging about both failed and successful experiments I've run on a Cyclone V FGPA but I'm not convinced there'd be a ton of interest. That said, I do not work at a company that uses FPGAs or proprietary hardware and I believe part of the reason the field is dark is because that's just how hardware development shops have operated.

Your experience of it being hard to even get started because of the insular hardware community mirrors mine. (Well, I guess my employer does use FPGAs somewhere, but in a department far from me.)

Are you just using a Cyclone V demo board? To be honest, it's actually a little hard to even find FPGA accelerators that aren't bespoke.

This might interest you: http://neuromorphic.eecs.utk.edu/publications/2014-07-14-dyn...

(Disclaimer --- I was a student working on this project)

Oh damn. Paywalled. It would be great if someone could throw their VHDL etc on Github

Are there performance numbers for this, on e.g. resnet-50?

Reply on news.ycombinator.com