That's not strictly true. An audio file with a single non-zero sample (usually set to full amplitude) is often used for testing -- usually called a Dirac impulse or similar.
That impulse will be (necessarily) band-passed by the playback hardware and put out filtered "white" noise.
That impulse can be recovered by a mic to show e.g., pre-ringing caused by (FIR) filters. An FFT of that impulse will show the playback hardware's response in the frequency domain vs full bandwidth.
> Thinking of the sample rate as a 'set frequency'
For e.g., a WAV file, that's a fixed number of samples per second (a frame being 1 sample x n channels). That is a set frequency, and deviating from it will alter the pitch of the music.
There really is no case where sample rate varies, unless we're talking about minute variations between the clock signals of different hardware, which requires the use of sample rate conversion to match.
A related concept is the bits-per-second of lossy formats (e.g., AAC) which may vary from frame to frame (and that frame will mean something different from a WAV frame).
> You need at least 2 samples to make an audible frequency.
I think you're confusing this with Nyquist being 1/2 of the sampling frequency. You can very much capture an audible signal with a single sample, but that signal will be limited (by hardware, by Nyquist, etc).
[Edit]
I should say that this single sample has to be non-zero and the playback system has to have a DC-offset that isn't equal to that sample's amplitude.