Billionaires didn't get to be billionaires by abandoning $10 million properties to avoid $1 million in taxes.