This stuff is already part of the training data - Laion for example (which almost everyone uses as training data) has a lot of csam in it (allegedly I never checked)

This has already happened with ChatGPT.