Top of the line Arria-10 FPGAs have about 1500 floating point MACs [1]. Using such a device, Intel claims ~1 TFLOP sustained for GEMM, the standard matrix multiply operation [2].
[1] https://www.altera.com/content/dam/altera-www/global/en_US/p...
[2] https://www.altera.com/content/dam/altera-www/global/en_US/p...