"In other words, there seems to be a mysterious divergence between people’s stated and revealed preferences. Why are we commissioning styles that we say we dislike?"

Oh, this is actually pretty obvious to explain. There are several equally big reasons, in no particular order here:

- Modernist/brutalist/bauhaus etc. building with any stated use type is on paper cheaper to build, because it removes all aesthetic features and many functional features - like rain pipes, slanted roof in favor of flat roof, human oriented features like benches etc. It will be much more expensive to maintain in long run, due to the environmental damage from missing functional features, but that's not architect's problem then.

- These buildings are build as a class warfare. Notice the stark difference between designs of private houses where higher class actually lives, and the designs which the are commissioning for the common public.

- Performative architecture. The weirder the art form, the better PR and resulting fame.