Selling it would mean you pay taxes. Retaining it you pay taxes. It's probably not going to be seized if you don't abandon it, it's not like it's a low income neighborhood in the way of a highway.
Billionaires didn't get to be billionaires by abandoning $10 million properties to avoid $1 million in taxes.