"it's unfortunately more about politics and luck than merit"
To reiterate, software is not all about writing software. There's usually an immense amount of buy-in you'll need from other people, not limited to other software engineers.
All while simultaneously trying to navigate budgets and legacy systems. Even if you're pretty heads down, you'll still see the outward effects of these things.
All these things are necessary though, as the system within which the software operates becomes increasingly complex over time, it becomes necessary to engage with many different people in a multitude of roles and capacities.