It's difficult to assess one's "comfort" with the math. I've been working with crypto for more then 10 years and I wouldn't say that I'm perfectly "comfortable" (e.g. the Ristretto stuff). Should I stop working with it?
Crypto is deep. You can get involved at the levels you feel comfortable with.