When you have so few bits, does it really make sense to invent a meaning for the bit positions? Just use an index into a "palette" of pre-determined numbers.
As a bonus, any operation can be replaced with a lookup into a nxn table.
As a bonus, any operation can be replaced with a lookup into a nxn table.
It seems quite wastful to have two zeros when you only have 4 bits it total
+0
-0
+1
-1
+inf
-inf Bits s exp m Value
-------------------
0000 0 00 0 +0
0001 0 00 1 +0.5
0010 0 01 0 +1
0011 0 01 1 +1.5
0100 0 10 0 +2
0101 0 10 1 +3
0110 0 11 0 +inf
0111 0 11 1 NaN
1000 1 00 0 -0
1001 1 00 1 -0.5
1010 1 01 0 -1
1011 1 01 1 -1.5
1100 1 10 0 -2
1101 1 10 1 -3
1110 1 11 0 -inf
1111 1 11 1 NaN ± 0 (infinitesimal)
± 10^-2n
± 10^-n
± 1 (unity)
± 10^n
± 10^2n
± infinity
But you still have all the other numbers carrying sign info. This is only the sign of denormals and that's way less valuable. Outside of particular equations it ends up added to something else and disappearing entirely. It would be way better to cut it and have either half the smallest existing positive value or double the largest existing value as a replacement. Or many other options.
Because of that, the best compromise when the weights are quantized to few levels is to place the points encoded by the numeric format used for the weights using a Gaussian function, instead of placing them uniformly on a logarithmic scale, like the usual floating-point formats attempt.