Certainly uncertain: When measurement results don’t match | Hacker News Reader

Certainly uncertain: When measurement results don’t match

twitteringmachines.com

8 pointsPascLeRasc6 comments

If I understand correctly, the audiophile review is claiming to recommend a network switch based on the difference of a few pico seconds of jitter!? Surely that is outside the range of human perceptibly right?

The main issue here is mixing up a statistically significant effect and that effect being significant. With a very high powered experiment you can detect very small effects but that doesn’t mean they have a significant effect on the outcome.

I really think the simple hearing test is best but should incorporate common tools of measurement science such as repeated tests and multiple, varied listeners. Audio “quality” is very subjective and can come down to preferences driven by culture, age, experiences, etc. Some sort of consensus rating over all reviewers should be done.

Edit: also the rule of 3s! Never measure twice, always 1 or 3 times, is particularly relevant to this article :)

Indeed, buffering the data by even just one or two sample periods and running the DAC from a local clock would be enough to eliminate any reasonable effect of network behavior. And it's probably already buffered by more than that, if data are sent as packets.

There's always the possibility that the equipment under test is itself badly designed, e.g., that the DAC is unstable against tiny changes in network behavior. That might tend to magnify tiny errors that would be accommodated by better designed but less esoteric equipment. Subjectively choosing among badly designed components could explain a lot, including why laypeople don't perceive the same effects when listening to mainstream gear.

One of the greatest misconceptions in all of Audiophiledom is the notion that measurements are objective.

Only if you regularly confuse empirical measures of your machine with subjective listener assessments of stuff you play through it -- which, as far as I can tell, is what the audiophile scene is defined by.

We can, should, and do make objective measurements of systems all the time. We then build a model (poorly or well) that attempts to relate changes in the system described by those measurements to changes in subjective perception by one or more listeners. At which point, a conscious human is in the loop, and all bets are off.

Every once in a while, what little of audiophile-woo that I have read almost reinvents double-blind psychometric studies. And then they don't quite get there, mostly.

I think many of the actors don’t want to get there. If you’re making digital interconnect cables made of 100% oxygen-free silver mined under the light of a harvest moon by Peruvian virgins, you probably want to take advantage of a vaguely scientific-sounding approach that isn’t fully scientific.

no word of a lie.

I looked up those switches fully expecting to see either AVB or TSN standards compliance. That could make a difference to listening in that an AVB/TSN switch preallocates fixed slots so that latency sensitive applications never wind up losing a packet due to network collision.

However, to see that these switches don't even support AVB/TSN makes me simply roll my eyes at the benchmarketing that audiophiles will swallow.

Reply on news.ycombinator.com