The upside is that if you only call a generic function with a u32, you don't instantiate an f32 as well. The downside is that when you do decide to call that function with an f32, all the comptime stuff suddenly gets compiled for the f32 and might have an error.
In practice, I feel that I gain way more from the fast compile than I lose from having a path that accidentally never got compiled as my unit tests almost always force those paths to be compiled at least once.