More recently they train on a mix of synthetic and organic text, like the Phi-4 and o1 / o3 models. Original copyrighted text can be safely replaced with synthetic standins.
Synthetic data will not replace original data like books. Synthetic data works very good for math.