But regardless of that, even if you buy cheap land in a low regulation county/state and use all non-union labor, the building cost is still more expensive than it would be if buildings had the reduction of costs that other durable goods have had over time.
Some of this is clearly regulatory, i.e. codes. Some is clearly consumer choice, note the failure of prefab and tilt up to catch on. Some is clearly ecology, note the disappearance of native large timber from most residential construction.
Many architects and builders have tried their hand at designing homes and construction processes that reduce the costs of buildings but if they have to build to modern codes, expectations and ecological standards they mostly see marginal improvements. There are just a lot of fundamental costs involved with site constructed buildings that meet modern standards.