No, when you are dead. Since you were once U.S. citizen in tax sense by owning U.S. properties and securities. (including you as permanent residents, or H1-B visa holders) The U.S. tax authority now has right to go to Swiss banks to request freezing of assets.
The worse part is when calculating your tax, the U.S. tax authority will treat you as a non citizen so they can tax assets more than 60k, instead of usually 1 million in 2009.
This just happens to one of the Taiwanese Billionaire, Wang Yung-ching. He owned U.S. properties and his company Formosa Plastics Corporation now is trying to estimate the amount of the tax they owe to U.S. tax authority, even the company is not a U.S. company and he has no U.S. citizenship and permanent resident status. But because both entities owns U.S. properties. The U.S. tax authority has their bites to all the company assets in Taiwan, too.
Meanwhile, his heirs in Taiwan still need to pay the inheritance taxes for their own shares of the inheritance.