Because the problem is that many programmers — almost all novices and even some pretty experienced ones — don't admit to themselves the simple fact, that they might be wrong in the what they are thinking right now, at the moment. Sure, they won't deny that they make mistakes sometimes, but yet somehow they are pretty self-confident while actually working. Basically, they think they are much smarter than they are. What's making the whole thing even worse is the fact that people (again, especially the novices) don't actually think about "how to make it cheaper (less painful)", they think about "how to code properly". Basically, "what should I do to not be laughed at, how do I seem smart?" They just know they must follow some "golden rules", so called "best practices". You know, the patterns, no goto, no code duplication ever. All that programmer etiquette.
That's why often instead of simple, short and not-elegant-on-purpose solution, non comprehendible, thousands of lines long abstractions are born. Programmer tries to justify it by that it "solves the problem we don't have yet", but usually it turns out it doesn't — which isn't a real surprise, as it's hard to solve the problem you don't know about yet. And while the crude and simple solution might have been rewritten in an hour, that not so perfect abstraction of yours requires days of works, and usually just keeps collecting hacks until it is so complicated that it's impossible to change anything without breaking something else.
Of course, the point is, as always, that you have to keep balance. But there is a strong bias towards "abstraction over duplication" and such, so it's better to push programmers towards simpler, imperfect solutions.