5,014 karma · joined June 8, 2017
2. If the comparison implements a strict total order, the output is sorted according to it.
What techniques?
The fact that invention is hard and actually you probably couldn't have invented a bunch of hard stuff is not really that, um, relevant.
Hy-MT2 also came out with a 1.25bit model using a technique called Sherry, where weights are trits with the additional constraint that exactly one trit in a group of four is 0. Four possible positions for the 0 times 2^3 possibility for the other three positions = 32 possibilities, so it fits in 5 bits exactly. You can also exploit the sign symmetry by factoring out a sign bit and you have sixteen possibilities for a group where the first non-zero trit is +1. Unpacking can then be done with a 16 bytes lookup table, which is small enough to do really fast with SIMD.
Their fork corrects the second inefficiency by using a group size of 128, but still uses 2-bit weights AFAICT.
It's possible to pack 5 trits into a byte, but the unpacking is not very efficient. Another recent idea is to add the constraint that exactly one weight in each group of four be zero, which gives exactly 32 possible states, so it fits in 5 bits.
Why?