Most of us would rather be creating art than be beholden to crappy commercial interests. Most of us would rather be the one giving orders than taking them, or compromising our art to other people.
We programmers have been incredibly fortunate to be considered "golden boys" (and a few girls). We spend no more time mastering our craft than the poet or the painter, but we get paid a lot more to do it. We still don't make as much as the one who brings in the seed money and keeps most of the profits, but because reasons we haven't been relegated to the barely-tolerated and barely-paid status of the people who write documentation or make our products attractive, no matter how good they are at it.
We programmers have, I think, become more associated with those who get rich off the process -- the CEOs and CTOs and such -- than the ones doing what we think of as grunt work rather than creative work. We get to dream that our side projects are just one elevator pitch away from our own C suite.
I'm suggesting that you're on to something considerably bigger than a personal gripe. This is a very widespread thing, and maybe the alternative to being unhappy about the difference between our side projects and our day jobs is to help everybody reconsider what "day job" really means. There are a lot of people who would like to be more creative in their work -- including collaborating creatively -- and perhaps finding that people would actually enjoy (and pay for) the results of that.
I don't think it's lack of regulations that makes our salaries high (barring the sky-high wages at FAANG). For example, in banking - a heavily regulated industry - coders still make way more than people who can write well ("poets"). I think it's just basic supply and demand, there's just not enough people good enough at coding compared to how many are currently needed across all industries. Let's enjoy it while it lasts.
Meanwhile, even pretty poor programmers can make a fair bit of money, rather than being priced out. But there are a ton of them, and honestly, there could be even more. Basic programming isn't that hard. Lots of people do it in high school, self-taught, and even more could do that given opportunity and encouragement.
I'm definitely enjoying it while it lasts. I've been expecting it to stop for three decades. I've been wrong the whole time and it looks like it will last the rest of my career.
But I'd really like to see us take a good long look at the whole thing. A lot of talent is going to waste. I'm happy to benefit, but wish others could, too.
This question would get the same response from me if posed by an aerospace engineer in the context of making planes for Boeing versus a home-built ultralight aircraft or drone. Sure, you don't get the same control, and you have to explain your choices and efforts to your employer, because you're not doing the same thing in those two situations.
If you want to find joy in programming, you need to own the project or find a project and organization aligned with what brings you joy. I wouldn't get any joy out of making some b2b financial services system, but I would get money from it so probably wouldn't object. I'd clock out at 4pm or whenever my 8 hours were up every day. OTOH, I did enjoy making and testing safety critical systems for aircraft. I wouldn't do it as a hobby, but I wouldn't object to longer hours to try and get things working.
> Isn't programming a more creative endeavour than we think?
This should be fairly obvious but programming isn't monolithic. Your experience will vary depending on your work environment and current project. So you can either put up with it (and there is still plenty of satisfaction that can be had from programming in such an environment, or from doing unglamorous work) or you can look for a job/company that operates differenly or works on more interesting problems (they're rare).
> you don't need to explain what you did to anyone in a standup. You can devote your energy to the programming instead.
These two aren't mutually exclusive. Working on a team comes with communication/meeting/status-report overhead, this doesn't need to take away from your productivity.
The wild west days of things can be crazy but fun. Then the crowds and tax collectors come, laws are made, and now these little atomized commits that stitch together APIs is "programming".
It makes me think of other areas of life. Most of us show up at the boring, overlegislated, agile test driven development phase of things. Maybe it is good to look for the wild roots of anything.