That's your problem right there. Many (most?) FOSS contributors are in it for the intellectual challenge. Designing a good UI is perceived as a lesser problem compared to technical challenges, and therefore disregarded.
Also, design is not the comfort zone of most FOSS contributors (which is why some FOSS projects do dedicated outreach programs not only to women or minorities, but also to designers).
Also, I think there is still some "it was hard to write, so it should be hard to use" mentality. I frequently see it justified it as an avoidance tactic for developer burnout: By making it hard to use your software (e.g. by providing no documentation, or by coating the documentation with a thick layer of jargon), the only users you'll get are the ones persistent enough to work through that on their own, and those will be the ones that deliver high-quality bug reports and contributions.