Hard for me to imagine that this could be solved in text space. I think the prediction task needs to be done on the audio.