Ultimately the debate here is “should richer people have to contribute back to society to help the lives of the poorer people”? Of course you can say no to that. But the majority of society doesn’t agree.
Poor people typically don’t live in the same neighborhoods as wealthy people in America. Wealthy neighborhood public schools tend to have lavish facilities from the tax base, so it’s more like “wealthy guy X doesn’t want to pay for the very fancy facilities he isn’t even using”, not “wealthy guy X doesn’t want to pay to send his poor neighbor’s kid to school at all”.
There are still schools rated much worse than others.
In my country your life is already decided at age 12- go to the wrong school and it's over.
I mean they may not use their local library or parks either, but they ain't getting a refund for that.
If you opt out and want your house to burn that is only fine so long as you ensure the fire doesn't spread, if there is signs it might spread out have to pay inflated rates for fire service to protect the other houses that you can't burn down with yours.