The answer is no, as illustrated by the Hotel paradox[0], in which we have a infinitely many rooms and want to accommodate a (possibly infinite) number of guests.. To summarize: For any finite number of guests, you can always find an even-numbered room to correspond to that guest (assume the guests are numbered sequentially, then double their number and put them in the room that has that number.). This creates a one-to-one correspondence, which means that the sets are the same size.
You might say, 'well, that only works because we're dealing with a finite number of guests. But we're talking about infinity'. There are a few different ways of answering that question. In my opinion, the easiest way to look at it is to remember that there is no such number as 'infinity' - when we say 'infinity', we're really trying to express the concept of growing without bound. So, the above strategy (double the person's number) works for any arbitrarily large group. At no point does it stop working, even as the group size grows larger and larger, so we can say that the two sets have the same size.
On the other hand, we have no such strategy for putting every irrational number in one-to-one correspondence with the counting numbers. That proof is a little harder, and the analogy with the hotel guests breaks down, unfortunately, so it's a bit tougher to explain.
[0] https://en.wikipedia.org/wiki/Hilbert's_paradox_of_the_Grand...
In short, infinities are complicated and intuition doesn't work well.
In slightly longer, you're talking about the difference of the rate of growth of two functions rather than actually about the cardinalities of the sets.
We define two sets as having the same cardinality when we can create a bijection between them. We can list the primes in order from smallest to largest and number them with the natural numbers. So we'll have 2 match up with 0, 3 with 1, 5 with 2, 7 with 3, etc. Every single prime number will correspond with a natural number AND every single natural number will correspond with a prime number, no exceptions. So they must be the same size. They are also the same size as the integers and the rational numbers but the set of real numbers is a bigger infinity.