If they use technology as we understand it, then presumably yes. Prime numbers are just a unique artifact of basic arithmetic. Our knowledge of prime numbers appears to date back to at least Ancient Egyptian times, so it's quite safe to assume an alien race capable of something as technologically adept as transmitting a signal with the purposeful intent of contacting another race would know what prime numbers are.
The issue is we're limited to our view point, we're exceptionally naive of anything different as we've never experienced it. So, is our technology the only way?
We're designing nanotechnology that could fabricate anything we could dream of, but what if an alien race ended up evolving a unique cell that could do it too. We have white blood cells, they have nanofab cells. Would they even comprehend math if they're just growing jet engines and rocket nozzles?
It's this latter, our unknowable unknowns that will broadside us. How do we relate to something that's unfathomably alien to us.
The good news is you and I likely won't be around to care when we find out what aliens are really like.