Rural America is, on the whole, losing population [1], and having the sorts of groups you're lumping move in (for either "hipster farming" or outdoor recreation reasons) provides socioeconomic diversification and robustness to many rural areas that would otherwise be in decline as family farms and natural resource extraction become less profitable.

[1]: http://www.ers.usda.gov/amber-waves/2016-june/five-years-of-...