250 is nonsense.
2xFMA per cycle @ ~4.5Ghz = 32*4.5 = ~144 Gflops
Beating cuBlas is unlikely. You probably made a mistake. Last I tested it, it was even better than MKL in efficiency.
Beating cuBlas is unlikely. You probably made a mistake. Last I tested it, it was even better than MKL in efficiency.
While flash memory became so fast in recent years, I haven't heard of any break-through technology prototypes to bring some significant progress into RAM. Let alone the RAM latency, which remained constant (+/- few ns) through all the years.
Unlike the main memory bandwidth or that of the shared L3 cache memory, the memory bandwidth of the non-shared L1 and L2 caches has been increased exactly in the same ratio as the FMA throughput. Almost all CPUs have always been able to do exactly the same number of FMA operations per clock cycle and loads from the L1 cache per clock cycle (simultaneously with typically only a half of that number, of stores to the L1 cache per clock cycle).
Had this not been true, the computational execution units of the cores would have become useless.
Fortunately, the solution of systems of linear equations and the multiplication of matrices are very frequent operations and these reuse most of their operands, so they can reach the maximum computational throughput.