There is so much wrong here that I'm surprised to see it even discussed.
3 karma · joined July 3, 2020
More specifically, I'm trying to use pitch (F0) to dynamically adjust the theta parameter in rotary positional embeddings, so the frequency of the positional encoding reflects the underlying pitch contour of the speech and instead of using a fixed unit circle (radius=1.0) for complex rotations, I'm trying to work out how to use variable radii derived from the pitch. The idea is to create acoustically-weighted positional encodings, where the position reflects the acoustic salience in the original audio. https://github.com/sine2pi/asr_model