I stopped resisting the distractions and just allowed myself to think about them while at the same time trying to focus on my breath. As the distraction faded my focus on my breath returned.
I've gotten better at this and I feel like I've rewired my mind into being able to maintain a "quiet mind" and self-awareness, at the same as talking to myself either in my mind or out loud. And so when I get anxiety I can stay grounded and now I just think about what is giving me anxiety instead of letting it go or forgetting it. It's helped me to get rid of generalized anxiety that I've been stuck with for a long time.