Conversely I've worked with lots of designers who also suck at UX.

I think the first paragraph nails it. The people who are good at UX are the people who have the benefit of testing designs with users and iterating on them. Either for the problem at hand or with the benefit of past experience.

As an industry I wish we could get past the idea that your job title denotes your ability in one area or another.