Notably newer Intel CPUs (iirc, Sandybridge and later) include a true random number generator, but I'm not sure how much software supports it yet.
The kernel developers just don't think it's a good idea to blindly trust unauditable hardware random number generators. Using it in conjunction with other sources of entropy is one thing; just using it on its own is putting a bit too much faith in it, especially because the code that uses it uses a generic wrapper for any hardware random number generation instruction, so if AMD or ARM or MIPS chips later come with random number generators, the kernel will be able to take advantage of their entropy without putting itself at risk if one of them is backdoored.
There's a difference between fearing that something is backdoored, and thinking that it's not a good policy not to blindly trust something.
This means it might take more time to generate the required number of random bits. So not trusting RDRAND means a process might block waiting for random data.
Then again, hardware generators may be affected by external forces - other than the NSA. For example, CPU temperature.
In Linux, there's a user-space deamon, rngd, that's responsible for mixing hardware random number generator entropy into the entropy pool. That's a more appropriate place to put the logic for adding entropy from RDRAND into the entropy pool; you probably don't want to be doing those 1022 iterations or 32 iterations with a sleep in between in the kernel itself.
The kernel just uses it as a parallel CSPRNG along with its own built in one. It runs its own, and RDRAND, and xors the results together. This allows it to get fast, good random numbers (which are important early in the boot sequence, for things like ASLR) even if its own entropy pool does not have very good entropy yet. This makes its random number generation at least as good as the stronger of RDRAND and the kernel's own CSPRNG. It does mean that if RDRAND (or some other processor's similar instruction, once other processors add it; it would be quite valuable on ARM and MIPS as embedded machines tend not to have very good entropy sources) is backdoored, and the kernel entropy pool has insufficient entropy, it may be possible for someone with knowledge of the backdoor to crack it, with a difficulty equal to the amount of entropy that the Linux entropy pool has. However, this is only true if you are using the non-blocking random number source, which tends to be used by things that need fast random numbers but don't need particularly good seed-grade entropy, like ASLR. If you need high-quality entropy, just use /dev/random and it will block until you have enough entropy to satisfy your request.
[1]: http://software.intel.com/en-us/blogs/2012/11/17/the-differe... [2]: http://software.intel.com/en-us/articles/intel-digital-rando...
On a related note, I noticed recently that the BCM2835 SoC in my Raspberry Pi has an RNG as well. I've been considering using it (off-line) to generate private keys for SSL certificates issued by our private CA.
/* the initial numbers generated are "less random" so will be discarded */
#define RNG_WARMUP_COUNT 0x40000