In other words, there can't be any commercial incentives that "align" with providing a perfect user experience.
The weird thing in my opinion: why is open source software such a pain to use? They don't have commercial interests?
Disagree, this is an unreasonable framing. We just need to exchange goods/services for money, not optimize every transaction on a curve by some measure of "extract everything"
There are many incentives apart from $ -- general fame, specific, niche credibility, love of the thing, internal projects (eg a phd/masters/capstone project), etc. Looking at everything from a $-incentive only will lead you to actions and incentives that don't make any sense. Look at the linux kernel or something like sqlite -- more deployed than almost any piece of "commercial" closed-source software, huge community, signifigant resources, but tiny, tiny slice of revenue by any metric of popularity.
In other words, you want the market to restrain itself, which of course never works.
You then observe people to have other motivations beyond greed. Yes! I emphatically agree. Only, those aren't subject to market logic other than being potential subjects of exploitation.
It's also evident that, as companies grow older and/or founders retire, they become abstracted from their investors/shareholders into money-making machines, and those will not accept a lesser return in exchange for societal good. Social responsibility should be baked into law, so companies don't have duty to their shareholders (AKA profit) as their maximum legal priority.
I don't think It'll happen in our lifetimes, but if humanity has any hope to last more centuries, it needs to seek what's beyond capitalism, hopefully before global warming forces our hand.
No one(execs, middle management, shareholders) is happy to earn a decent(10-20%) profit anymore and call it a day.
Every last drop needs to be wrung out of the customer or the employee or the citizen in the case of government.
My guess is its a combination of 1) lack of resources and 2) being developed for the developers themselves (who are weird users).
Most of the best open source software (e.g. Linux) is that way because commercial interests have incentive to pour some of their vast resources into it (and if copyright was ever abolished, expect it to start sucking, because the main reason we get to seee the commercially-funded improvements is the genius of the GPL).
Clearly, open source software with a good interface becomes more popular. I suspect, being popular isn't really a good experience for the developer, who then gets inundated with unreasonable requests etc.
When they put their individual egos above the actual outcome of their efforts, well, that's a programmed disaster of course.
Even the GPL isn't needed anymore when you can trivially reverse engineer nearly anything.