When you say "birth rate is falling" its effectively the second derivative on "number of people" right? As in, if we have go from 2 children born per person to 1.5 children born per person, we are still growing the number of people.
Even if "children per person" falls slow enough you could even indefinitely increase the rate that new people are added right?