On the one hand money in an endowment has presumably already been taxed at least once. Taxing it again is double taxation and that’s bad because once you allow money to be taxed twice, you have to be ok with it being taxed many times over — O(1) to O(n) if you will. The moral position of government would be that taxing the outcome of growth and wealth creation is fine, but retaxing existing money is the government breaking its covenant with the people.
On the other hand I would accept the government’s position if it were to say aloud that there goal is for all large pools of money to be reclaimed by society. Double taxing wealth is then ok because it will, eventually, redistribute the wealth. In such a moral code, a $51bn pile of gold is quite an aberration.
It is classic right versus left: natural hierarchy versus we’re all the same.