The idea is rather than storing a number x as (exponent, mantissa), just store (log x) as a fixed precision number. Multiplying two such numbers is just addition, dividing is just subtraction. TBH I didn't read the article, but my reaction is that yes, that works, but one must sum all those products, and now summing becomes an expensive operation. Maybe the total cost saves area and power, but it beggars belief that it is 10x more efficient. They must be doing PR math: our low precision log scheme is 10x more efficient than a higher precision traditional approach.
Another thing to keep in mind is a lot of inference is done using very low precision math and so the cost of doing multiplies isn't that bad. Yes, it is still (n bits) squared, but as n gets small, n^2 still isn't too bad.