On the one hand, I can trivially demonstrate they are less numerous than the set of integers
On the other hand, Cantor's exercises for showing larger cardinality can be exercised equally on the set of prime numbers
At least as far as I've been able to see. This shows a contradiction that can be easily accommodated for with multi-dimensional or an otherwise richer descriptor set. Any refutation that starts like "assume there exist a function f that generates all the primes as a sequence of integers" I reject. The whole point is that I'm not assuming that - maybe a proof that no such function can exist for the set of primes is an undiscovered property.
If you decide to be extremely disagreeable and stubbornly throw out any speculative, at least I'm left with only with the contradiction. They seem to behave way closer to the irrationals then anything else.
I think a more useful exercise is whether the peano successor function can be defined. For the irrationals, I think the answer is "not a chance". For the primes the intuition is "maybe" but when you start stating why, nearly every argument you give (statistically you can do it - with enough information you can do it, it is a discrete number and so on) you can defend for the irrationals with the same reasons.