It does increase naturally. The catch is: If the overall gene pool is too small, the viability of successive generations will decrease faster than the rate at which the gene pool can expand.