Hacker News Reader

I thought the same when I saw that the financial industry was hiring FPGA people for low-latency algorithms.

My understanding (as a non-FPGA expert) is that currently FPGA beats generic hardware (CPU,GPU) for "small size algorithm" (i.e that do not need GB of weights), while enabling a certain flexibility vs ASIC.

My guess is that you cannot bake all the weights into the circuit topology, so you are still bound by the memory transfer speed (to be double checked).

View on HN

The value proposition I am pursuing is not faster speed, but increased throughput (and possibly reduced kWh).

I'm trying to understand the benefits of streaming expert weights through hardware that offloads the math and avoids storing all of the weights in RAM at once.

But that's not the only thing that can be streamed and offloaded. '

Finally, I'm trying to come up with approaches to reuse old hardware, old GPUs, old RAM instead of paying today's prices for GPU VRAM or unified memory. Even if I do not end up showing any particular FPGA benefit, I might be able to better run very large models on systems without GPUs or without unified RAM.

Reply on news.ycombinator.com