This paternity worry seems to be a leftover of agrarian societies where it is important to ensure inheritance. In tribal societies, as we lived for millenia, it was much less of a worry as raising the kids was a tribal concern.

There is a famous quote from a Montagnais Native American to a Jesuit priest that tries to tell him that he should prevent his wife from having other relationships or he would risk bringing up another mans child: "Thou hast no sense. You french people love only your own children; but we love all the children of our tribe" - https://sexgendersoc.files.wordpress.com/2012/09/4-montagnai...