Analysing bird songs with Wigner transform | Hacker News Reader

Analysing bird songs with Wigner transform

soundshader.github.io

76 pointsgbh444g22 comments

Couple points of gentle critique:

It's kinda hard to compare the different spectral representations when they're zoomed and cropped differently.

Spectrograms can be misleading, in a few different ways. Magnitude FFTs discard phase, which we can hear. And our eyes tend to fixate on the peaks, but the noise floor between harmonics in speech had a big impact on perceived quality. Choice of color scheme and gradient changes how we look at the spectrogram: they can emphasize mathematical or coding artifacts we wouldn't hear, or hide things which we can hear. At the end of the day, we don't hear with our eyes... So a spectrogram is a tool for looking at audio, but not always an 'honest' one. So I'm a bit suspect of pouring over slightly different spectrograms, and worrying about which ones look better aesthetically.

In which cases can you hear phase? I know you can hear phase from interference where there are two sources but I didn't know you can hear phase of a single audio source.

Phase changes over time of single a single tone are audible. If there is a linearly changing tone and the phase is not tracking exactly (I believe it is square, I'd need to brush up on my chirp math), this will color the chirp in an audible way.

Also the relative phase of multiple tones affects what the actual shape looks like. A classic example is a square wave. Yes, it needs all odd harmonics at a sinc(f) magnitude, but it also needs all of those harmonics at specific phases.

Here's an experiment which makes it very obvious that we hear phase.

Take some audio signal X and apply the FFT with a 50% overlap between frames. Now randomize (or zero out) the phase, invert back to time-domain, and then take the FFT again and invert again back to the time domain. The resulting signal has 'consistent' phase between frames due to the extra round trip, but will still sound terrible. (It's equivalent to a single round of Griffin-Lim phase reconstruction. You can run more iterations to get something that sounds better, but it's still not perfect even if you run an arbitrary number of iterations.)

This proves that the information in the magnitude spectrogram is a subset of what we pick up with our ears: There are signals with the same magnitude spectrogram which sound different.

If you're every out and wonder which bird is behind the song you can hear, this is a great app: https://birdnet.cornell.edu/ (Android and IOS)

You record a portion of the song, and it uses machine learning to analyse it and tell you the bird with a confidence figure. Works really well.

Birdnet is great!

Once you've identified the bird, you can then listen to a variety of additional recordings on https://www.xeno-canto.org/ which I believe is one of the sources used to train the machine learning model.

As an FYI, if you are interested in the fundamental frequency of birdsong the GitHup repo below might be of interest. It is an STFT + interpolation to get an accurate (potentially quickly changing) frequency estimate: https://github.com/JorenSix/stft_freq

Was this work done on data stored in lossy formats? The appendix with the talk about format conversion makes it sound like it. Should this not have been the first thing to be avoided („garbage in, …“)?

Yeah, it was mostly mp3 and ogg files. If you're suggesting flac would be better, I don't think flac adds much clarity over a decent 320 kbps mp3.

>Was this work done on data stored in lossy formats?

You're right to mention this. Lossless audio is preferred for analyst software. Even good MP3s tend to top out around 16kHz.

The quality of the recording will also be dependent on the microphones used and their frequency range.

When we have analysed animal sounds it's useful to play the sample slower, pitched down. Having those higher frequencies recorded well, above 16kHz, make a huge difference to the signals information.

Getting data from the real world always results in noise and artefacts etc. If you've done any kind of work with data, a large part is preprocessing. MP3 seems like a very minor source of problems compared to the potential others. Road noise, other birds, lorries backing up, people talking, mislabeling etc.

The difference however is that formats like MP3 work on the principle of psycho-acoustics, which means they modify the original audio in a way that is meant to be indistinguishable by human hearing, but it could very well cause a difference in the spectrum in places where it matters for the analysis (it can add and remove frequencies).

The compression is not wholesale adding or removing frequencies, it is selectively lowering the magnitude and SNR. Overall, the RMSE of 320 kbps mp3 against lossless of many common and simple waveforms is very low. Certainly it is acceptable for something like birdsong measurement.

That's a great point, hadn't considered that. You changed my opinion.

On the other hand it’s not as if bird songs are encoded binary information. They’re complex to our ears, but probably hold up pretty well under common audio compression algorithms.

Why would one bother what effects psychoacoustic models might or might not have on your data when you could simply circumvent the problem completely?

You are right in the abstract, but these are very high-resolution "lossy" signals. If you look at the spectral response of the bird songs it falls well inside the part where there is no signal loss. Your complain is like bothering about a face recognition system because it used lossily compressed but very high-resolution images. That criticism would be just as nonsensical.

A more meaningful criticism would be the use of the wigner transform itself, which seems to produce ringing artifacts in the visualization, not seen on the more common windowed fft.

You assume the author has access to lossless sources, or has the funding and means to produce the recordings themselves. I don't have this impression. You work with what you have.

https://freesound.org/search/?q=blackbird

(Filter by WAV, FLAC, or AIFF.)

Try a more obscure bird species, like the one in the original article.

https://freesound.org/search/?q=Eastern+Wood+Pewee

1 result

https://www.xeno-canto.org/explore?query=Eastern%20Wood%20Pe...

228 results

Furthermore, your search for blackbird on Freesound gives 639 results based on a text match, while xeno-canto gives 7449 results from 32 species, with metadata tagging it by species, location, date, time of day.

Thank you, that's a wonderful website!

Also see https://news.ycombinator.com/item?id=27073944

Reply on news.ycombinator.com