Have you tried using a purpose built pair programming app? Essentially these are tools with low latency screen sharing and remote control. Then you can pair-prompt in a call.
https://zed.dev/docs/collaboration/overview
Not sure if the agent stuff is shared in the "channels", though.