Comment by tacos | Hacker News Reader

Comment by tacos | Hacker News Reader

So, so many words. But you can stop reading right here:

"The track was from a CD and therefore suffers from compression issues. That is, some of the frequencies may be missing due to the method employed to fit everything on the CD"

Programmers dicking around with audio and getting it all wrong is the new programmers dicking around with graphic design/UX and getting it all wrong.

FFTs really don't work for this stuff. There's not enough frequency resolution at the low end. Even higher up the most common error is an octave error. This can be caused by even harmonic distortion. Also, the brain is great at creating a missing fundamental. Mixing engineers use this to clear out the low frequency mud from a mix.

Tinkering with AutoTune for even a few minutes you'll quickly learn that the name itself is a lie. It doesn't Auto-matically Tune anything. It generates a crude, crappy attempt at tracking pitch (and monophonic only pitch) which then requires a trained musician to wiggle against using a mouse in a tiny window for hours and hours. Get it wrong and it sounds like a robot. Most people get it wrong. "Wrong" has become a sound. Now people get it wrong on purpose.

Melodyne is more interesting but also trips over itself.

This is a hard, hard problem. Even if you can find the loudest frequency and map it to a note, it doesn't mean it's the note in question.

I think the difficulty in audio engineering is in the fact that audio and music in particular are such cultural constructions. Take a modern dance track back a thousand years and it wouldn't be a hit; it would probably get you killed as a witch. No wonder Autotune is so hard to get "right".

When you find the loudest frequency and map it to a note, it is most definitely the note in question (in the mathematical sense). But cultural and stylistic expectations of the human brain might not agree with the math.

That's why I think somewhat surprisingly audio engineering will be a human career as long as music will be a human art. You can't say that about many automatable professions.

Sounds like trying write software to identify and enhance the most "important" elements of paintings, or to clarify the theme of each stanza of a poem.

You accurately describe the many subtleties in audio engineering and pitch perception, but you omit the conclusion that, at least for pathological cases, there is no objective "note in question." Two reasonable people, both musically educated, can have different interpretations of a note, especially in the context of a particular chord or musical arrangement.

Agree, with caveats: in the context of vocal tuning there's often a desired note that represents the melody. And objectively there's often a note that a musician intended to play.

Even if the instrument and signal path cooperate to capture this intent (the inharmonicity of the string itself; nonlinearities of the microphone and amplifiers; resonances in the instrument soundboards and rooms themselves) you're already chasing ghosts!

Then add in a mixing engineer boosting and reducing frequencies, a tape machine varying in speed ever so slightly, and a mastering compression step throwing away amplitude detail -- and you've really got to wonder what the hell someone thought they were doing by running FFTs on a 50 year old analog mix and pretending they've got insight into the original solo instruments.

FFTs actually work for this kind of stuff but not in the way he's trying to apply it.

Instead of trying to reconstruct the sound directly you could use the data for getting hints at which instruments (by looking at their particular characteristic spectral distribution) were involved. From there you could then try to "fill" the observed frequency pattern with the instruments involved at the notes that would seem more plausible.

Still much harder to do than to say but that's what comes to my mind.

If anyone's ever interested in doing this kind of analysis I can't recommend this tool [1] enough, is it one of the most easy to use yet very versatile and helpful when dealing with these kind of problems. It is also free.

[1]: SPEAR, http://www.klingbeil.com/spear/

The tradeoff on a FFT is time vs frequency. So you can have either accurate time data or accurate frequency information. If two notes cross a time window you're going to get the note in between. But if you shrink the time, then you don't get enough frequency resolution. Because pitch is logarithmic (the difference between C1 and D1 is less than 2Hz; C4 and D4 is 32Hz) FFTs really don't work that well for this sort of stuff. Short, low notes in particular are a killer. And the loss of bucket resolution down low makes detection of the fundamental very difficult. The big bump in the wigglegrams that tool provides are not particularly useful for the specific task at hand.

What? No... you can approximate both data and frequency as much as you want to.

What tacos is trying to say is that in an FFT, you will have the same number of samples in the time domain and the frequency domain (weeeeellll, I'm simplifying a bit, half of those 'frequency domain' samples are actually phase information, so you really have half as many frequency domain samples as you do time domain). So, if you wish to have a high resolution in the frequency domain, implying lots of samples, you're going to have a total sample time that is quite long, and you won't be able to say exactly when each of the different frequencies was actually present in the sound.

Take a concrete example. Let's assume that I have a recording in PCM at 44.1kHz, and I want to see all of the frequencies present, to a resolution of 10Hz. Nyquist tells us that we only have 22.05kHz of signal present in the sample (well, one can hope, otherwise if the recording didn't filter out everything above that when coding to PCM, we're going to have a mess of aliasing going on, which is outside the scope of this post - just assume the person doing the encoding knew what they were doing, mkay...). So, 22.05kHz. And we want a resolution of 10Hz, so we're going to need 22.05kHz / 10Hz = 2205 samples. Of course, we actually need double that in the time domain (we throw away the phase information remember), so we need 4410 samples in the time domain to get our 2205 samples in the frequency domain. 4410 samples at 44.1kHz sampling is equal to 0.1 seconds.

If I decide that I want a better frequency resolution, say down to 1Hz, then you quickly arrive at the conclusion that you will need a full second of samples. In other words, your resolution in the time domain has decreased as you improve the resolution in the frequency domain.

You can improve a bit on this situation by using a sliding window with a carefully chosen windowing function. The frequency information will still be smeared out in time, but the windowing function reduces the amplitude of a frequency the further it is from the centre of the sampling window, which sharpens up the smear a bit.

Most music analysis tools use a sliding window combined with a windowing function to correct for most of this. You'll still have your frequencies smeared a little in time if you're plotting f against t, but the windowing helps limit the smear.

Reply on news.ycombinator.com