Every source I can find says that secularism/atheism is growing worldwide. Not sure what you're talking about.

Even if "most" people don't "defect", you have to make up the remainder by people going the other way, for which there doesn't seem to be any evidence.