Most stats have shown we haven't really made much labor efficiencies on housing construction. No one wants to live in a prefabricated / factory built home, which is what would drive down the labor costs dramatically. That said I'm kind of a weirdo and like to browse the container home websites frequently, lol.
Further, modern homes are bigger (and nicer) than our parents or their parents generation lived in.
Couple that with the land component being basically inflationary due to natural scarcity PLUS government zoning rules restricting lot sizes to be artificially high, banning multi-family construction, etc.
You have a lot (basically all) tipping the needle towards higher cost.