Actually for many neural network apps 8, 4 even 1 bit is sufficient for representing weights.