I'm missing something here. If you collect data up to 48kHz, then there are still 100,000 or so instructions occurring in each sample. Getting from there to breaking RSA should have some explaining.
And the answer given in that link is: "First, when the CPU is carrying out a long operation, it may create a characteristic acoustic spectral signature: for example, below we show how RSA signature/decryption sounds different for different secret keys. Second, we get temporal information about the length of each operation, and this can be used to mount timing attacks (see Q10), especially when the attacker can affect the input to the operation (i.e., in chosen-ciphertext attack scenario)."
An RSA encryption requires doing the same operation over and over again, with some inputs the same each time, for much longer than 100,000 instructions.
What kind of relationship is there between sample rate and the frequencies you can record?
I may remember it wrong, but I believe you need at least double the sampling rate vs the frequency you want to record. And even then it's a very crude representation of the acoustic signal (Basically a one-bit resolution).
As snippyhollow pointed out, you're right with regard to frequency and that is the Nyquist-Shannon sampling theorem. However, it is important to realize that the resolution of the sampled signal is separate from the sampling rate and so the second part of your statement is incorrect. In theory, a signal sampled at its Nyquist-Shannon rate may be perfectly reconstructed to its analog equivalent. In practice, quantization (the number of bits used to represent each digital sample) will cause distortion to occur, but the bit depth or resolution of a sampled signal is only limited by hardware - in other words, there is certainly a limitation but one-bit quantization would not be useful (and thus not used) for recording an ultrasound signal.
You're thinking about the Shannon's theorem. Here it is about knowing what operations the CPU is performing/looping on. This frequency is not related directly to the frequency at which the CPU operates: it's related to the frequency at which the plates of the capacitor oscillates (which is related to the frequency at which the CPU draws power and so operates...).