I agree with you here. As a designer, I have helped out FLOSS projects with UX design and usability. However, the environment is such that if you would suggest UX could be improved, the response often is that devs feel insulted because you are dissing their code (which is not the case, but they don't see that).
On the other hand, taking a look at WordPress, they invested quite some time in user testing and improving UX by professionals, and that makes it one of the most usable web publishing platforms out there.
So, it would be too easy to say all FLOSS is bad at UX, but most developers don't even see it as an actual discipline, and hence do not see a problem.
Nicer devs, that's the key :)