I also think that aliens might well have completely different mathematics. To me, the most compelling reason to think that is that the development of mathematics is rooted in our conscious perception of the world around us. Our conscious perception is rooted in how our brains work, how our world looks, and how the stimuli from the world are connected to the brain.
Consider for example the theory of differential geometry. Would we care about objects such as curves, surfaces, and manifolds if they (well, their low dimensional special cases) weren't ubiquitous in our world?
Would we care about permutations and symmetry if some people hadn't played around with arranging sets of objects?
Would we care about waves (sonic, fluid, E&M) if we didn't observe water flowing?
Even problems that seem to be mere puzzles with no basis in the ``real world'' are inextricably linked to the conscious perceptions that we learned to form by living in the world.
There was a great push to axiomatize all of mathematics beginning in the early 20th century and now it can be said that a lot of mathematics really is a collection of maps from assumptions to statements.
It could be that alien mathematicians have reached a collection of maps isomorphic to the one we have, but they ascribe completely different meanings to them.
Another possible scenario is that within the universe of all mathematics, our conscious perceptions and cognitive biases have carved out some subset, and alien mathematicians have carved out another. These two subsets could be so vastly different that they are mutually unintelligible, and very difficult (maybe impossible) to translate between the two mathematical languages. Even if we could translate the alien mathematics to a symbolic language familiar to us, and check that their theorems are correct, because of the vast difference in meanings ascribed to these two sets of mathematics, we just might not be able to understand what the alien mathematics means!