The model was trained with features of human voice bound to a frequency range so it may work for "cross-language" sync. Why not give it a go and check the quality? It won't change the content of original segments but only shift them along the timeline if there are gaps.