Not rich people, but people in
consumerist societies.
Consumerism as a way of life is hostile to having children. Consumerism construes all of life, and the purpose of life, as consumption and in hedonistic terms. It means that sexual intercourse and sexual relationships are perversely construed in terms of consumption. A sign of this is the 20th century normalization of contraception, where the intrinsically procreative end of sexual intercourse is intentionally and explicitly blocked and frustrated in order to center pleasure (the irony of hedonism is that it actually makes sex less enjoyable, as pleasure sought for its own sake undermines itself, but I digress). In a healthy sexual relationship, children are a welcome result of sexual union. In a consumerist relationship, they are an impediment to hedonistic consumerist indulgence, so much so that abortion is sought when contraception fails. Even the notion of lifelong marriage stands in the way of the consumerist juggernaut, as commitment to one's spouse means denying yourself countless, admittedly shallow and selfish sexual encounters. Consumerism is an agent of social and individual decay.
Furthermore, children cost money. The money you could spend on consumption, on that new Mercedes, may need to go toward the good of a child, which, according to a consumerist calculus, is lost. And how can you keep up with the Jones' when the measure of wealth aren't children, but more stuff? The poor may have had more children in consumerist societies, because various luxuries are out of their reach anyway, so the dilemma doesn't manifest as much. Also, the fish tends to rot from the head down, so it takes a while for the habits of those further up the economic ladder to trickle down.
Of course, not everyone must have children, but healthy societies do tend to have many. If it were simply the case of the former, we would not see the hostility we see toward the latter.