Comment by svat | Hacker News Reader

Comment by svat | Hacker News Reader

This is a great idea; I really enjoy all these "two channels simultaneously" (side-by-side translations, video with subtitles, and in this case video with a readable transcript, where you can scroll in the video or scroll in the transcript, and be synchronized).

I had done something like this a couple of years ago for some specific set of videos (e.g. https://shreevatsa.net/tex/program/videos/s10/ — compare with https://youtubetranscript.com/?v=_0Cv1G_s4gQ for the same video), but never got around to making it general; glad someone has done it. It takes just a few lines of Javascript, using the Youtube API, to do this i.e. keeping the video and text in sync (just view source on either page to see the JS at the bottom).

Something like this can also help with audio recordings (generating the alignment automatically is called "forced alignment" and there are tools like "aeneas" for this). In case anyone's interested or wants to help (for Sanskrit texts): see https://github.com/shreevatsa/web-align-audio-text deployed at https://shreevatsa.net/ramayana/sarga/ and better version at https://github.com/avinashvarna/audio_alignment deployed at https://avinashvarna.github.io/audio_alignment/

This is cool!

We're doing forced alignment with audio recordings i.e. podcasts. Here's an example from a test client: https://www.withfanfare.com/p/seldon-crisis/search-by-the-fo...

Grateful for any feedback you might have.

Also, if you run a knowledge-dense podcast, or know somebody that does, I would love to talk to you/them. I'm for example considering to link places, people and things in a transcript.

Lovely, thanks for the example. I really like how the underline is subtle yet effective. The animation/effect where it seems to inch forward is also intriguing; how does that work? (Also, this is subjective but have you considered making double-clicking on the text open the audio player even when it's not been started yet? Would make it more discoverable, though I imagine some could get annoyed. Another minor thing: when the audio is paused and you double click somewhere, the audio position changes but this is not reflected in the player; only takes effect when you unpause.)

You're doing cool work, good luck with your service! It looks very useful and if someone I know is running a podcast I'll recommend it for sure; it looks really well-done and polished (at least the example you shared).

Thank you! I wrote your points down. We're still learning how to make it most usable.

We know the start and end timestamps on a word level and we know the current player time. So all we do is set a CSS class on the words that are currently playing. We not only highlight the current word, but also the words that are close in time. This is what generates the effect.

We just released autoscroll, which is something that was requested often.

Wow, this looks incredible! Love how the underline travels and guides the eye.. Very impressive!

Excuse the question, but "forced alignment" is when you don't have timestamps, like in webvtt?

Thank you!

Yes, exactly. We do forced alignment when you edit your transcript. The new words don't have any timestamps, so we need to align them. For short sections we use interpolation. If we need align whole sections we use Gentle[^1].

[^1]: https://github.com/lowerquality/gentle

Thank you, this is really interesting! :)

Reply on news.ycombinator.com