Isn’t this the point though? Does it actually require that much gear to get measurements that are accurate enough that no human can perceive the difference? With harmonic distortion for instance, we know what is inaudible and yet people keep lauding companies that push ever farther past it.
Does it actually require that much gear to get
measurements that are accurate enough that no
human can perceive the difference?
For speakers, yes, you need some of this sophisticated gear if you want to do objective (as opposed to purely subjective) reviews.
The Klippel NFS is an automated rig that measures a speaker's output from many different angles and uses some nifty math to subtract the room's influence, giving results more reliable than a physical anechoic chamber. That rig costs something like $100K; I forget.
Getting a 360 degree picture of a speaker's sound output is pretty important. The majority of sound we hear indoors is reflected. You are, essentially, hearing the sum of what your speaker is doing at every single angle all at once so if we're going to characterize a speaker's performance in an objective way we need something like that.
Of course, you don't need need a $100K rig. You can measure a speaker in your suburban back yard (or better, an open field) from a few different angles and get a pretty decent approximation of all of that. I've done it quite a few times! All you need is a $60 calibrated mic, a protracter, a tape measure, and some free software. And the willingness to annoy your neighbors with loud tone sweeps. Usually takes me a few hours to measure a speaker; most of the time spent is just dragging things around so I can do 20 minutes of measurements from 5 or 6 different angles.
However, if you're looking to do this with large numbers of speakers and have lots of money like Amir does, the Klippel certainly starts to look attractive.
For measuring DACs and amps, no, I guess you don't need a professionally calibrated AP analyzer. I think you can spend a couple hundred bucks and get pretty close. Maybe less.
For measuring headphones, yeah, I think you need something like the GRAS headphone measurement rigs that are $6K-$13K. Or maybe the $299 one from MiniDSP is just as good. I don't know!
The calibrated microphone is good for correcting some rough low frequency room reflections and frequency response unevenness and measuring spatial response, but not really measuring other distortion and noise. It's only typically -60 to -80 dB SNR. Even though distortion adds, it'll end up in the noise, and cheap ADC plus microphone preamplifier are typically not as high quality as DAC and speaker amplifier either adding its own cruft.
The headphone measurement couplers are more of an art than science for the moment. You can, in fact, make a decent-ish rig for a thousand or so, but MiniDSP one is not it. For plain equalization I'd trust your own ears with sine tones and some bandlimited noise instead. For measurements, it's just poor.
Klippel NFS is something I have personally seen exactly once, and I cannot attest the quality of its measurements (esp. room subtraction) personally. Almost nobody has it.
You also need a crane to hoist the speaker 20 meters into the air, or you have ground reflections to contend with. (You could bury the speaker so its front is flush with th ground for a hlaf-space measurement and do some math with that, of course.)
https://www.minidsp.com/applications/acoustic-measurements/l...
There are limits to that, admittedly. You're still going to get that bounce below a certain frequency (height dependent) so you'll still want to get it off of the ground as high as possible. At 2 meters you can get accurate measurements down to 90hz-ish IIRC. Actual in-room bass response is highly room dependent anyway so that's not totally the end of the world, but of course it's not ideal, and overall it goes to show you how impressive and valuable the Klippel is.