Development processes and software architecture follow from business process and architecture... it's hard to be agile and develop services with clean separation of responsibilities when business insists on monolithic hairball project reqs with fixed deadlines.
(aka Conway's Law: https://en.wikipedia.org/wiki/Conway%27s_law )
Nobody says they want to waste money building bad software. The problem is that the factors which ensure it are political and hard to change. It's easy to forget that since we tend to focus on the visible technical aspects but they're almost always a reflection of the environment.
For one thing, I've seen each generation of new managers forget or ignore the knowledge hard-won by their predecessors or the software body of knowledge. Oh, and ignore their current experts, too. I think it's a bug in the human wetware because it happens so often.
For another, it was explained to me when consulting at $giantcorp that sometimes - for example, regulatory compliance or competitive edge - it's more important to get a shitty monolith out there, bugs and all, by a drop-dead date than to save in the long run by doing a good job.
And as long as there are people out there willing to work for low wages fixing or rewriting the pile of crap, and it's profitable for $company, the practice will continue.
Or until someone can prove, with cost measurements on multiple large-scale projects implementing the same requirements, that the ROI - in bottom line $ terms - on a well-engineered system is much greater than the crappy equivalent.
I can't see that happening because of all the variables involved (team, skill, chance, cost, variance in interpretation of requirements, and so on), plus, who would pay for that?
EDIT - typo.