What qualifies as progress is a matter of opinion, but generally speaking, denser cities produce more rapid technological innovation.
I have no way of knowing whether the new mystery tech would make dwellings taller, or cheaper, or more environmentally friendly, or more mobile, or more self-sufficient, or better workspaces. It's the sort of thing that is only obvious in hindsight. It may already be invented, but does not yet have a practical and economical application.
Whatever it is, it might require a cartel enforcer to mandate it in order to yield greater benefit for the society as a whole. You're not being oppressed when the city forces you to give up your well and septic system and connect to the municipal water and sewer. In theory, you're achieving greater gains through (enforced) cooperation with your neighbors. In practice, you may be oppressed by official corruption in such a way that it wipes that common benefit out.
And luxury is relative. The poorest person in the U.S. can have a better defecation experience in a public park toilet than the richest king in the world could have had on his private, hand-carved night-stool 500 years ago. One day soon, it might be considered luxury to crap in a hole in the dirt... on Mars--as the alternative would be to do it in a pressurized closet, into a machine that recycles the water out of it and turns the rest into agricultural compost.
I didn't intend to make a point about building codes, but rather about how quickly humanity changes its standards for technological advancement. For tens of thousands of years, a mud hut with tile roof and underfloor heating would have been a luxury accommodation. Now, for a lot of people, it's rustic and primitive, fit only for camping trips rather than permanent habitation.
Humanity is scary not only for its ability to construct shelters like this without any specialty-evolved biology, like a woodpecker's beak, a beaver's teeth, or mole claws, but because it can extrapolate the idea of shelter so far beyond our current survival needs that even a thing that well exceeds the minimum requirements is still considered obsolete.
I agree that we take learn to things for granted quickly. I think that's unfortunate and leads to a lot of unhappiness. Forcing our addiction to needless luxuries on others is a significant way that it leads to unhappiness, but far from the only one. You also see people dying of lack of physical activity, for example.
I don't think it makes sense to talk about "the intent of a building code". Since we have not yet produced artificial general intelligence, building codes are not yet sentient entities with intent. Different people who supported the enaction of the building code had a wide variety of intents; the different people who convert it from a piece of paper into a pattern of human behavior, by enforcing it, also have a wide variety of intents. Most of these are unknowable.
For the most part, it is very easy to determine the original intent of a building code, especially relative to some other varieties of laws.
The enforcers of the building code sometimes betray that intent. This is unfortunate, but there is little that a legislator can do to control a cop or county inspector that may be separated from the consideration of the bill by time, distance, wisdom, and attitude.
Building codes do little good if they are not updated frequently, and appropriate for economic conditions. Earthquake-safe architecture in all residential buildings is not appropriate for Haiti, but it may be for vital infrastructure, such as water treatment plants, ports, power plants, and hospitals.
Thanks to changes in U.S. economic conditions, it may be necessary to revise building codes to make it easier for people to buy new homes costing $40k or less, rather than go deep into debt for an existing home with median price $230k. It does no good to mandate by law that the worst possible home that can be built is still too expensive for anyone to buy.
It should be obvious that one guy building shelters from scratch in the outback is not going to significantly affect the housing market. But there's always that one guy, that power-tripping bureaucrat out to make an example of someone, who ruins it. The problem is not the building codes. It's that guy, who uses them as a club to enforce conformity, rather than as a standard which may accommodate reasonable variances.
It's because of that guy that building codes should not have the force of law. You just can't put the power to knock someone else's house down in the hands of a jerk or an idiot.
First, it treats people as individuals unaffected by social forces, whose power is treated as unrelated to how they choose to use it. In reality, the dynamics of power in a social organization is an arbitrarily complicated topic, and deeply constrains how people who have power can use it; and people's opinions mostly arise from the conventional wisdom of their social groups, which is subject not only to the limits of bounded rationality, but also to memetic selection pressure.
Second, the legislative and regulatory process is far from the disinterested consideration of the common good you ascribe to it; it's a conflict-filled social process involving constant negotiation among regulators, legislators, and their constituents. It is a mistake to try to understand it as if it were a sentient entity attempting to achieve certain goals by choosing means that its beliefs predict will promote those goals. To a significant extent, the individual people acting inside of it are such entities; but their goals and beliefs are at odds with one another, and as a result, the overall system does not behave in such a fashion. Sometimes it may happen to do so, which is often because the relevant part of the system is dominated by one or another powerful elite that is sufficiently unified to behave that way — although often the interests it promotes are not the common good, but a narrower set of interests. More often, due to collective action problems, it thrashes around in ways that appear irrational and random if you attempt to understand them using the metaphor of a sentient entity.
Your puzzlement about how we could have ended up with such perverse regulations as those that gave rise to low-flow toilets is a good example, but in fact public choice theory abounds with examples.
I say these are contradictory opposites because the first amounts to treating human beings as isolated contemplators, sort of floating in a void, while the second amounts to treating them as mere cells in a superorganism, devoid of individual beliefs or interests, their individuality entirely subjugated to the goals of the collective — a sort of Hegelian apotheosis through what seems to me to be the ultimate degradation. (Perhaps with the occasional exception of that guy.)
I think that with a somewhat more nuanced theory of human collective action, we can understand what does and doesn't work about existing forms of social organization, and then collaborate to improve them, providing us with alternatives that allow everyone to flourish in ways out of the reach of hunter-gatherers building seismically-unsafe huts, out of reach of proletarians toiling in misery in Foxconn factories, and even out of the reach of Captains of Industry hoping fondly to turn our nation into a new Somalia or Honduras.