Beauty is in the eye of the beholder though.
Some people might say that various religious monuments remind them of how that religion suppresses individual expression. On the other hand, the same might be said about certain architecture in say Japan, that it evokes a sense of collectivism. Yet some people will find that beautiful while others might find it stifling.
I would guess that the vast majority of people have never heard of "Brutalism", and if you were to show them examples there would be a wide range of responses. How many people could say whether something was influenced more by Brutalism vs Modernism vs Minimalism?
Take the Guggenheim. It was not well received when first revealed, but is now considered a prime example of Modernism, and even Brutalism, though I expect many everyday people would be still divided on whether it's beautiful or not.
Beauty is subjective and multidimensional, so when you say " utilitarianism at the expense of beauty", what I hear is "utilitarianism at the expense of a particular aesthetic of my preference."
An example that many people would be familiar with is IKEA. Yes, some people do find the Modernism of IKEA to be sad or lifeless, but others find pleasure in the simplicity and clever functionality.
I haven't even discussed how cultural and historical contexts influence these perceptions, and how that will differ depending on the audience. I think this plays a significant role in coloring our judgments of Brutalism.
Anyway, my point is just that it's not so simple, people define and find beauty in many different ways.