ParentFull threadlucidrains·I think what they did in the paper is to increase the chances by multiple rounds of hashing, up to 8 times. They show experimentally that 8 times was good enough to be equivalent to a regular transformer.View on HN