It could just be using speech to text (e.g. Whisper) on your input, and then using its text model on the text of your words. Or has OpenAI said that they aren't doing this?
> Advanced voice uses natively multimodal models, such as GPT-4o, which means that it directly “hears” and generates audio, providing for more natural, real-time conversations that pick up on non-verbal cues, such as the speed you’re talking, and can respond with emotion.
From https://help.openai.com/en/articles/8400625-voice-mode-faq
If you want to try only voice, Try unmute.sh by Kyutai which will be eventually open-sourced
I'm still interested in what keyword I could use to search for the latest research in voice models.