They should tear down the Fed Mint building behind that Safeway and turn it into housing. I don't even believe that branch does anything other than print proofs. A waste of space, but it's not happening during this presidency for sure.
One easy to appreciate example are universities & other schools holding (& sometimes leasing) land adjacent to them. If they didn't, in the distant future if they need to expand, some scalper or holdout could cause large problems.
And as a consequence of the old adage "time is money", an indefinitely existing organization has an indefinite amount of money. Those institutions you mentioned have no issue buying whatever they want; holdouts be damned. A government pretty much has infinite money, churches are quite wealthy as a whole, and universities have a substantial pool of donors to tap despite having insanely large endowments that are quite well managed.
I don't think your argument holds up.