However it's still wild to me how fast and responsive it is. I can talk for 10 seconds and then in ~500ms I see the updates. Perhaps it doesn't even transcribe and rather feeds the audio to a multimodal llm along with whatever tasks it already knows about? Or maybe it's transcribing live as you talk and when you stop it sends it to the llm.
Anyone have a sense of what model they might be using?
I want to say 300ms which would coincide with your 500ms example
I am also too lazy to google or AI it but it’s something I remember from when I taught ESL long ago.
LLM to types and done