Not sure wealth is moving out or rather people who cannot afford to live there are moving. People who are wealthy and afford to stay will stay in the coastal cities where most of the outbound migration is happening.
The article largely discusses the band from Charleston to Jax. That's coastal. There is a lot of wealth flooding to Charleston and areas like Beaufort. I have seen it first-hand, although I live on the west coast.
There are other factors at play besides just affordability.