Hacker News Reader

The middle ground here would be an FPGA, but I belive you would need a very expensive one to implement an LLM on it.

FPGAs would be less efficient than GPUs.

FPGAs don’t scale if they did all GPUs would’ve been replaced by FPGAs for graphics a long time ago.

You use an FPGA when spinning a custom ASIC doesn’t makes financial sense and generic processor such as a CPU or GPU is overkill.

Arguably the middle ground here are TPUs, just taking the most efficient parts of a “GPU” when it comes to these workloads but still relying on memory access in every step of the computation.

I thought it was because the number logic elements in a GPU is orders of magnitude higher than in a FPGA, rather than just processing speed. And GPU processing is inherently parallel so the GPU beats the FPGA just based on transistor count.

With FPGA you are sacrificing performance for flexibility you are far less efficient in transistors for any given task than with a dedicated ASIC even if it’s a general compute ASIC like a GPU is today.

The reason no one is building large FPGAs is that there is no market for them.

If an H200 scale FPGA was viable we would have one.

Reply on news.ycombinator.com