I also had to do this in my previous work and I took the phonetic embeddings of reference and transcribed text and ran a dynamic time warping with them.