What I think we figured out is that we have "compatible" (co-enabling) anxieties that keep our relationship mostly functional. That feels like a drastic over-simplification but accurate, I think. All I know is it feels like the most emotionally productive year of my life and our relationship is way better now. However, I still haven't figured out how to tone down the anxiety much :)
Good news, you're completely normal. Every couple has their unconscious dance both in a positive and negative way.
Barring physical ailments, low vitamin D (etc), the solution for anxiety is do your healing work. Anxiety is often unresolved fear, anger, shame, guilt, etc. Everyone has anxieties, and they have to be dealt with daily.
That distinguishes between "I truly want to be alone" and "I want to be alone because I am not the person I wish I were".