arjf

Most likely microphone lower range limitation. The fundamental frequency is out of the microphone’s range, so you only see the harmonics.

32.7 Hz is C1, lowest C on a standard piano (A440 tuning).

No inherent limitation of the technique, the resonators can be tuned to much lower frequencies. But indeed you‘ll need a better mic than the iphone’s to get frequencies below 50Hz iirc… too bad - the watermelon ripeness checker could be a really cool app :-)

Thanks - adding all of that to my to read / to do list :-)

Thanks - to be fair the electric piano seems to me like a relatively favorable case for this approach.

A tracking resonator bank should self-tune to any frequencies... so as long as the density of resonators is adequate, after convergence, it should paint a representative picture of the tone profile. Then you can try funny chords, or see how harmonics interfere, or see what happens when you hit 2 adjacent keys, etc.

Fun analysis experiments like this are why I made the free demo app (it runs on iPhone/iPad/Mac):

https://alexandrefrancois.org/Oscillators/

I think the output of a tracking resonator bank is only the basis for higher level analysis that will produce results suitable for specific applications (see my comment on frequency component tracking and prediction/feedback).

I used the Exponentially Weighted Moving Average (aka low-pass filter) because it has a very nice iterative form and is very computationally efficient. My objective was low-latency for real-time systems (so no looking into the future either). I haven't looked into using other types of filters because I haven't felt the need for my own applications.

Also my primary objective was tonal analysis so that's where I focused my limited time and resources.

I haven't had time to explore what to do with broadband transients much. A tracking resonator bank will certainly capture the energy (either in tracking mode or not). To me the synthesis examples I have posted on the project site sounds very comparable to traditional vocoder results; not bad but not great, especially with transients (as expected...)

From an analysis point of view, I anticipate that a Novelty measure computed from a tracking resonator bank would be quite usable...

My approach is to let the "bins" collide at the filter bank level (really let nearby tracking resonators agree on the dominant frequency in the neighborhood), and use the bank's instantaneous state as input for a frequency component tracker, whose output is a list of (frequency, amplitude) rather than an array of bins.

Here is a short video demonstrating the concept (with spectrogram-style visualization): https://youtu.be/STayypC1pvU

This is all pointing towards a dynamic systems approach, with prediction/feedback loops, e.g. establishing a tonal context and feeding it back into the analysis.

I believe some plasticity in the natural frequencies in the bank and tuning of the resonator dynamics would improve the convergence time to some extent, but I think this will only go so far and most of those effects should be addressed via prediction/feedback.

I envision the timbre analysis to take place on these tracked components as well as harmonics should be tracked as components whose frequencies are multiples of a fundamental (so analysis on actual small number of actual frequencies rather than a large number of bins).

Sounds really interesting! Could you share some description of the algorithm used for chord detection? What model of tonality are you using for pitch/chord naming?

Spectral analysis has indeed been around as a concept for centuries and there have been apps based on the FFT for decades, so definitely nothing new there. What I have implemented however, while based in known concepts and techniques, allows to achieve real-time, low latency and high resolution (both in time and frequency dimensions) performance that I believe are out of reach of established (published) methods. The apps you link are most likely making use of the FFT, which has become widely supported with efficient hardware acceleration and easy to use libraries, because of its central role in ubiquitous DSP applications, e.g. compression. I would be interested in any publications or at least technical descriptions of algorithms/systems that achieve similar performance!

Actually digging into SWIFT a bit more, the formulas differ by more than just the heuristic for alpha (unless I missed something) so the analysis in the SWIFT paper does not apply directly to(or maybe even at all).

Just want to call out the resources listed at the bottom of the Resonate website:

- The Oscillators app demonstrates real-time linear, log and Mel scale spectrograms, as well as derived audio features such as chromagrams and MFCCs https://alexandrefrancois.org/Oscillators/

- The Resonate Youtube playlist features video captures of real-time demonstrations. https://www.youtube.com/playlist?list=PLVcB_ABiKC_cbemxXUUJX...

- The open source Oscillators Swift package contains reference implementations in Swift and C++.https://github.com/alexandrefrancois/Oscillators

- The open source python module noFFT provides python and C++ implementations of Resonate functions and Jupyter notebooks illustrating their use in offline settings. https://github.com/alexandrefrancois/noFFT

The Sliding Windowed Infinite Fourier Transform (SWIFT) has very similar math, and they provide some analysis in the paper. I use a different heuristic for alpha so I am not sure the analysis transfers directly. In my upcoming paper I have some numerical experiments and graphs that show resonator response across the range.

It only requires more computation if you really need to compute the full FFT with all the bins, in which case the FFT is more efficient... With this approach you only compute the bins you really need, without having to pre-filter your signal, or performing additional computations on the FFT result. Some sliding window FFT methods compute frequency bands independently, but they do require buffering and I really wanted to avoid that.

Interesting - thanks for sharing!

This formulation is close to that of the Sliding Windowed Infinite Fourier Transform (SWIFT), of which I became aware only yesterday.

For me the main motivation developing Resonate was for interactive systems: very simple, no buffering, no window... Also, no need to compute all the FFT bins so in that sense more efficient!

Indeed... I honestly don't remember where or how I sourced the value, and why I did not use the "correct" one - I will correct in the next release of the package. Thanks for pointing it out!