Exactly as xnornet was doing way back in 2016 - shadow 32bit weights, quantized to 1 bit during the forward pass.

https://arxiv.org/abs/1603.05279

I personally have a pretty negative opinion of the bitnet paper.