Except the place is being filled with rich people. Who drive up the prices for everything else.

Soviet Union had the same problem: in a vast country being 99% shitland, there were very few livable spots where everyone wanted to live. They solved it by basically implementing an internal immigration barrier: only way to live in such a place was either for the Party to send you there, or to get married to a local. In the end it is either that, or simple pricing out of those who can't afford, unless you can make more places in your country livable.