I would say training a CNN from scratch or even fine-tuning one takes a lot of domain knowledge and best practices which often are not standardised yet. Besides, we don't even know why they generalise in the first place! See:
https://arxiv.org/abs/1611.03530,
https://arxiv.org/abs/1711.11561