If I try to concentrate on my breathing I always feel like I'm not getting enough air. If I persist, I honestly start panicking a little bit.

Related: I can't snorkel for the same reason, except I panic A LOT.