In some cities. In other cities, that's just not true. Example: in Atlanta, GA, the median income is right over $55k. According to Zillow, that will get you a house in the $227k range. There are tons of single-family houses with yards in that price range (or even half that) in the city.
I ran similar math for Dallas, TX. I'd bet there are a lot of markets where this is true.
Housing is only expensive in the expensive housing markets.