It’s not like they do it on purpose rubbing hands
Which will tend to be less desirable areas, by definition.
One approach, of course, would be to also make existing houses less expensive/desirable. Notably, wealthy people tend to have the resources to fight efforts to do that.
Despite what many in the UK think about it "being full", this is mostly an illusion from how cramped are the parts in which the houses are allowed to go; the country as a whole is mostly undeveloped.
(There are of course other considerations, with "why did you build on a flood plain?" being one of the failure modes in the UK).
Building on farmland has national security concerns (as countries are slowly realizing), as the country ends up dependent on foreign food imports after awhile. Can't grow food effectively when there is a subdivision there instead eh?
In most high demand areas (which are the ones everyone is complaining about, generally), all the decent land (or even just 'normal' land) is already developed.
Bonus - next time around, even the flood plain land might be 'too expensive' to develop on, and folks will only be able to afford houses at the dump, or in a swamp, or the like. Yay.
I've seen that mentality in other parts of the world. The thing is that agricultural land is often protected because it is good agricultural land. It is literally the reason why the cities were built where they were built, since that land provided for the cities. Unfortunately, good agricultural land also tends to be easy to build on and we have a tendency to ignore features that may make it a poor decision to build on. (Such as the flood plains you mentioned.)
If a bunch of good people acting in good faith produce a result an evil mastermind would want does it matter if they are good? On many levels obviously yes, on some practical levels the system and the incentives still need to change.