It's interesting that they lowered the misalignment rate by that much with only 3m tokens of training.

Maybe we can align models by ourselves to our liking in the future.