The thing is, with Linux development, these things tend to work their way in _gradually_, rather than totally radical rewrites. (If you recall, I tried the radical rewrite thing with my Zinc replacement [1] for the crypto API, which wasn't too well received.) So I'm definitely not looking to cause a ruckus, but I do really want to get this into good shape. Provably secure constructions and slow-but-steady work will hopefully be key in doing that.
So, that may not the concrete answer perhaps you were hoping for, but that's a snapshot of my general approach.
The use of RDRAND was especially overzealous. The problem is always finding the line between retaining entropy and performance - in this case (without having done the statistics) I believe you have done the right thing.
If I wanted to contribute an entropy mixer not based on CRC32 (as the current one is) that is much faster, includes test vectors and entropy analysis - where would be a good place to start?