I'm speaking as one of those kids who could always do maths and that's how I do it, maybe there's another way but a lot of people I've spoken to who is the same agrees, with some exceptions. I view it as a skyscraper each level of learning requires the underneath level to be solid, if each level is a little wonky, then as you get higher and higher it can all come crashing down. Once you know how though,and have worked through the problems a few times, then use the calculator by all means.
An example: addition is commutative, if you know your sums, then this is pretty obvious, so when you do group theory, commutation is obvious, because you know 1+2 = 2+1, and you wonder why this is an issue, if you don't know you're addition tables then commutation is a strange thing, you can see it's something thats useful but why's it a big deal. Same with the identity element, then you can move onto rings and fields with the same level of understanding. If you don't know the basics in your bones then each of these concepts gets a little more rubbery.
Edit: or maybe look at it this way - if you're a writer and don't know the rules of grammar in your bones you'd be a lesser writer than some who does. Why would maths be any different?
From my personal life, at one point I thought I wanted to be a chemist. I loved the theory to the point that I enjoyed reading the organic chemistry text book. But I also had to take analytic chemistry which was five credit hours and two days of lab a week and I only got a D by setting the curve on the written final...and the way I did that was by reading the analytic chemistry textbook cover to cover in the three preceding days, I had long since stopped going to lab.
In retrospect (retrospect at the time), I realized that I had always hated science lab. The reason I hated it was because science labs are designed around getting the correct answer that is already known...i.e. the teacher wrote down the composition of the unknown beforehand. [1]
Later having spent time in creative studio environments, I can rule out that my dislike was from the shared space. What I love about studio environments is people are being creative and feeding off of each other. Sure it's competitive, but the competition belongs to the players and wowing and being wowed by one's peers is different from the rote exercise that is an undergraduate science lab.
So I never became a chemist and that's because I never really had a personality for a career as a scientist. It's almost certainly a good thing for me that I did not. Which does not mean that is the case for everyone.
Anyway, the two things are not orthogonal. A person can strive to develop grunt work discipline in a field in which they have no interest in pursuing further. The discipline has value because even my neighbor the painter gets up some mornings and paints paintings to sell because, you know, it's money.
Good luck.
[1]: Looking at it now, it seems absurd that the standard practice involves keeping knowledge secret.
I've felt for the better part of three decades that my colleagues who were proficient in maths had superior capabilities and this has always been reflected in disparities in pay and promotion.
I took early retirement and returned to uni to learn something new... After all these years I'm still struggling with maths but if there is one thing I've learned with near certainty is that the learning process all but demands working out all the maths with pen & paper. I think it's just part of the way the human brain works.
* I have also learned along the way that the math that I took time to work out stayed on in my memory for longer.
* Sometimes, the beauty of the algorithm is in the details.
* By repeating a certain exercise for different numbers, I have come upon special cases of the problem that enhanced my understanding of the problem.
So... decide what sort of work you want to do, and plan accordingly.