> its only a bubble when people want to sell or refinance though

The "everyone will just hold forever" theory doesn't describe markets well. People usually move because they have to: unemployment, run out of space, divorce, death, marriage, etc. This will not change, even if the rate of turnover is slower.