This is an overgeneralization. By the fourth century it became standard to forbid marriage because was believed that previously permitted marriage was only for individuals who remained celibate within the marriage.
I find this to be unlikely.
This position also ignores the East, as we tend to do, although I will admit they understood themselves to be changing convention when permitting it.