Biggest underlying reason for why software ends up complex, is because real world domain in which software operates, is also complex.
If engineers want the business to pay them for simplifying maintenance, they can identify a collection of support tickets that have a compelling root cause, and propose refactoring that system to eliminate the bugs and maintenance issues.
What I usually see is us engineers calling it “tech debt” and complaining we never take time for it, which is about as effective as the bookkeeper complaining about the paperwork that debt payments create for him. Businesses love debt. It’s a useful financial tool. The business doesn’t get the issue.
We also increased profit by lowering our runtime cost by some of those optimizations.
A large part of my career has been telling my boss "we should invest 5h of cleanup time here to make sure that in the future we won't have those 5h worth of obstacles here before building a new feature on top of it".
In industries where requirements change on a weekly or monthly basis, agility is worth a lot.
So, indirectly, yes: you can get promoted and collect revenue by simplifying and refactoring.
Most of the complexity and bugs I see in software are not because of the problem domain, but rather because of over-abstraction, under-abstraction and abstraction leaks, and also because of limitations and complexities introduced by the programming model or environment.
(unless you consider that "supporting five operating systems and the language must be X" is part of "essential complexity")
Of course, the more complex your domain is, the bigger is the program. But the non-essential complexity that exists due to the bureaucracy of languages/libraries/frameworks is a much bigger factor in adding complexity, bug and lines of code. Some examples:
- Manual allocation and deallocation of memory is a good example of something that we might think as essential, since it's intertwined with our domain code, but turns out to be unnecessary (even though the replacement has downsides). The billion-dollar problem (nulls) is another one.
- Supporting multiple environments/browsers/platforms. Competition is good, but the cost is steep for third-party developers. Using multiplatform frameworks partially solves but also has drawbacks: performance, limitations, bugs, leaky abstractions. If you need to be closer to metal, then different OSs have different threading models or API styles. Sometimes they don't even expose the main loop to you. You need to work around those limitations.
- In most environments we still don't have a nice way of handling async operations without leaking the abstraction. The current solution is adding "isLoading" data everywhere (or creating some abstraction around both buttons and the fetching mechanism). Concurrent Mode in React is probably the best thing we have so far.
- Most modern Javascript toolchains need multiple parsing steps: in the transpiler (to convert to ES5), in the bundler (to detect dependencies), in the linter, in the prettifier, and in tests. Compatibility between them is not guaranteed, and you might get conflicts which have to be solved by finding a middle ground, and that sometimes take more time than writing features.
- Dogmatism is another issue. I remember in one workplace years ago there was a "ORM only" rule and most of us would work the SQL and then convert to Rails ActiveRecord (or worse: Arel). In the end it was a complete waste of time and the results were impossible to maintain.
- I also think that the old Peter Norvig quote that "design patterns are missing language features" still stands. Go has proven that it's possible to have "dumb, simple code", but in other languages our best practices involve adding non-essential complexity to products.
The only exception to that in my experience is SQL: if a query is too big is not due to some bureaucracy of the language, but rather due to the complexity of the domain itself.
For example, wrong abstractions. As long as engineers writing the software are competent (if they aren't - that's a totally different story) they'll try to chose right level of abstractions for current understanding of the problem. Years down the road, a lot of their choices maybe end up to be a mistake, very often because understanding of the problem changed, or problems itself changed, and something that was nice and clean solution isn't one anymore. If problem isn't fully fixed and 100% understood, you'll never be able to make all the right, future-proof decisions about right abstractions.
About abstraction, even when it's good, it still depends. For example: you can use a Visitor Pattern to solve a collision detection problem, but if the language had multiple dispatch you would have less complexity. So it is definitely non-essential complexity.