GPUs are optimized for matrices of floating point values, so current neural networks use this as a basis (with matrices containing the scalar weights).