Go to Wikipedia and search for "Demographics of X" where X is a nation from the list given in the above-linked article: Australia, Austria, Canada, Czech Republic, Finland, Ireland, the Netherlands, New Zealand, and Switzerland.
(Nation/ Total Fertility Rate) (Australia/ 1.97) (Austria/ 1.41) (Canada/ 1.58) (Czech Republic/ 1.49) (Finland/ 1.85) (Ireland/ 2.1) (Netherlands/ 1.66) (New Zealand/ 2.1) (Switzerland/ 1.46)
Those numbers are for the years 2007 to 2009, depending on the nation. Note that only Ireland and New Zealand are at replacement levels. Residual Catholic natalism may explain Ireland's current fertility (given their loss of religion, don't expect that to last). Reading between the lines in the Wikipedia article for New Zealand, it seems that the Maori and Pacific Islanders are doing more than their fair share of baby-making. My guess is that they are more religious than the contracepting whites.
Bottom line: Lack of religion is either non-adaptive from a fitness perspective, or it is strongly tied to traits which are themselves non-adaptive.
UPDATE: I can't get the formatting right on that table of fertility rates.