Was following these two projects by someuser on Github which makes similar things possible with Local models. Sending screenshot to openai is expensive , if done every few seconds or minutes.
https://github.com/KoljaB/LocalAIVoiceChat
While the below one uses openai - don't see why it can't be replaced with above project and local mode.