Do you think Pollen is applicable to distributed AI inference? I think it could work for realtlime Voice Agents running directly on mobile hardware.

There are speech-to-speech LLMs that are big and do pure audio in audio out. But you can also make voice agents that use multiple smaller models cascadded. ASR for transcription, LLM for response text, TTS for speech, interrupt detection. If you try to load ASR, LLM, and TTS models that actually do a good job onto the same mobile device all at once, you can't get it to be realtime. But if you run them in a distributed setup, where each device has only one model loaded and streams its output to the next task device, you might achieve realtime performance while using stronger models for each task.

Does this sound possible, or am I misunderstanding how Pollen works?