Stop and think of it not in terms or whether it reflects badly on you but on the consequences for society. There is nothing good that can be said for creating enclaves of poverty.
By keeping the rich away from the poor we can avoid the harms of inequality. We can have poor neighborhoods, rich neighborhoods, etc, each of which has negligible inequality.
Do you disagree with the premise that local inequality is worse than global?
Poor people have no choice but to live with it while the more resourceful can afford to isolate and protect themselves but that's not an equal society or one that aspires to be one. No one can advocate that one should not try to get the best for themselves and their children but the if the bare basics of education, healthcare and safety are reliant on socio economic background that there is little incentive for those in power who are always from a privileged socioeconomic background to actually provide meaningful equality beyond posturing.
I think there is a danger of an exclusionary way of thinking that encourages ghettos and the kind of entitled thinking often seen here itself on hacker news that presumes in spite of all these disadvantages somehow all these children should compete fairly and succeed with their more privileged counterparts and if they don't and get stuck in a cycle of poverty they only have themselves to blame.
This kind of thinking leads to a bigoted society, and you then can't complain when you or your children are similarly excluded by those from a higher social-economic background in neighborhoods and other aspects of life, when not possible legally by financial or social means. This doesn't produce an enlightened and egalitarian society, you reap what you sow.
Ah, see, I'm not a parent, so who knows, my perspective may well change when I am, but if you asked me what a parent's job is, that is not the first thing I would say. I think I would instead say something hand-wavy about raising valuable, high functioning, members of adult society. Certainly, safety is a prerequisite to making it into adult society, but it seems like quite a lot of the stuff parents do to try to provide their child with "the best, safest, environment that they are reasonably able to provide" is counter to the main goal of raising good people. Keeping kids segregated from "those people" (whoever that happens to be in the parents' view) is a perfect example.