Tangent, but I suspect you've researched this topic more than I have -- do you know of any good source on the subject of FPGA NNs?
It dawns on me that FPGAs might actually be better than GPUs for a project I've been thinking about, but I know relatively little about state-of-the-art design/synthesis techniques.