Depends what you mean by "enough".

24MHz is "enough" for 20 bits@96kHz ADC and some post processing.

But 20 bits@96kHz is not decent.

For reasonable SNR, you need at least 24 bits, and even then "the experts" offload to an external CPU http://www.tested.com/tech/pcs/454839-tested-why-high-end-pc...

For signal (less audio are more "controller") with high precision you need micro controllers with the power of at least an early 2000s PC (several hundred MHz and single cycle mul/div).

Raspberry Pi 3 is close, but it needs an external ADC/DAC.