The top 3% aren't buying homes every year.