I shared the same confusion before, but I do have some ideas:
I think you could establish a communication protocol with an alien species with something like this:
First, establish the meaning of a set of mathematical symbols by choosing recognisable / universal mathematical entities and demonstrating the symbolic relationship between them
That means things like eg. a picture of euler's identity and then symbolic representations beside it, then varying the individual elements and showing the results.
This isn't easy (if you've ever played that boardgame about inferring another player's rules with pyramids, you'll know what I mean...) but it is probably doable.
And you might be able to symbolically demonstrate Peano Arithmetic or something.
Once you've established a vocabulary of mathematical symbols, you can maybe use that to pin down a basic physical vocabulary. You could maybe identify different elements based on eg. their atomic number. You can establish a time vocabulary by giving a formula for the orbit of their nearest star and asking them to solve for x. You can describe spatial relationships between things.
It seems very difficult to go from there to a yet higher level vocabulary - but who knows, it may be possible.