That's certainly how I hear it, too. I've never heard a woman expressing fear that she might have waited too long, such that her child will get Downs. Rather, fear that she might have simply waited too long, and now it is too late.
So for non-fundamentalist women, there is fear of having a Downs child, but it gets rolled into the package of "worried about getting to have a successful pregnancy at all".