Your statement is only true for CPU operations. Modern GPUs have their hardware and drivers optimized for floating point operations.