You don't need an ADC, you need a fast threshold circuit with regular sampling that will turn the noise into a bitstream, and a shift + latch register to accumulate large enough words from these bits.

The biggest problem of such a design might be correlation with the variations of the power supply voltage.