That's not even close to being true. As long as you're paying taxes where you're living (and most foreign jurisdictions are quite prickly about such things), if the taxes you pay to the foreign government exceed what you'd owe the US government, you generally don't have to pay any US income tax[0].
[0] https://www.americansabroad.org/us-taxes-abroad-for-dummies-...
Edit: Apparently, while the income tax situation isn't too bad for expats, the US has reporting requirements for foreign banks that are rather onerous. As such, many American expats have a hard time getting bank/investment accounts abroad, which is what OP was talking about.