See the late Tim Bryce's Theory P essay: https://www.phmainstreet.com/mba/ss050117.pdf and note that he uses the exact same "artists vs. house painters" analogy. Before his retirement Tim had been managing large software projects since before much of Hackernews was born, and had forgotten more than most internet commenters know about what it takes to manage a successful project.
The "software businesses where true creation is happening" are few and far between -- Apple, maybe Google at its peak, and a handful of others. They are run by geniuses, typically from top-tier schools like MIT and Stanford. They are not the typical software organization. The typical software organization produces the software that keeps civilization humming -- and doing that is a proven, teachable, scientific process. Lately we've seen many organizations that should be doing this instead cosplaying as a bunch of genius disruptors, but they have nowhere near the brainpower to actually pull it off, and the world has gone to hell in a handbasket as a result.