Hacker News Reader

Comment by Sophira | Hacker News Reader

To be clear, it's not LLMs creating the captions. Whisper[0], one of the best of its kind currently, is a speech recognition model, not a large language model. It's trained on audio, not text, and it can run on your mobile phone.

It's still AI, of course. But there is distinction between it and an LLM.

[0] https://github.com/openai/whisper/blob/main/model-card.md

View on HN

It’s an encoder-decoder transformer trained on audio (language?) and transcription.

Seems kinda weird for it not to meet the definition in a tautological way even if it’s not the typical sense or doesn’t tend to be used for autoregressive token generation?

Is it Transformer-based? If not then it's a different beast architecturally.

Audio models tend to be based more on convolutional layers than Transformers in my experience.

The openai/whisper repo and paper referenced by the model card seem to be saying it's transformer based.

Whisper is an encoder decoder transformer. The input is audio spectrograms, the output is text tokens. It is an improvement over old school transcription methods because it’s trained on audio transcripts, so it makes contextually plausible predictions.

Idk what the definition of an LLM is but it’s indisputable that the technology behind whisper is a close cousin to text decoders like gpt. Imo the more important question is how these things are used in the UX. Decoders don’t have to be annoying, that is a product choice.

Whisper is a great random word generator when you use it on italian!

Reply on news.ycombinator.com