The number of neurons firing does matter, but what matters even more is how they all are connected.
I personally think once we get adequate algorithms and data structures out there, the metric will switch to numbers of lookup operations performed, and floating point operations will just be a multiplicative constant in front of the whole complexity formula.