you make fair points. However the most common advice you get is , "focus on user experience". Most domain experts focus more on the tech as opposed to users. Also some of the great/successful startups have been started and run buy people who've never done it before.
A domain expert scratching their own itch often is unwittingly focussing on UX of the target audience, which is one of the reasons that open source tools by and for developers that don't have a lot of deliberate focus on UX beyond what works for the people building it often are fairly good, while the same kind of things built by developers in more consumer-oriented markets often have very bad UX.
Ultimately, its the same principle as dogfooding -- the most effective way to really understand UX deeply is for the developer to be the target user.