Replace housing, with salary and you will see why your argument isn't entirely right.
Should salaries/wages increase over time? yes. Should labor and employees remain affordable to employers? yes. Are both possible? yes.
The right question is who is paying, and what are they paying for.
Plus at the end housing is subject to the same laws of economics. Demand, supply equations, timing, value with regards to location and proximity to city enters, school districts etc.
You can't talk of home prices from one dimension alone.