We let them do whatever they want with it, because doing so has been advantageous for the rest of us. At the end of the day, rights are recognised by law, law is created and upheld by people, and people can change their minds.
I can imagine a future where we say "actually, this multigenerational NGO shouldn't use this much wealth to carry out this flawed mission, let's do something about it." Doing something wouldn't even have to mean taking their money by force: consider how the work of the Ford Foundation differs from the views of Henry Ford himself, and how that could have happened.