This should be used for high entropy tokens during pre-training.