And as Dylan Patel notes in his reply [1], none of it matters if it isn’t measured doing real work. Comparing the cost of hardware matmul is the same type of thinking that lead to prediction of Nvidia’s quick demise in the face of numerous AI chip startups. It’s just matrix multiples after all!
[1] https://twitter.com/dylan522p/status/1647362670801170433