For the use cases I know, some quality loss wouldn't be to much of an issue, but at least rudimentary video would be desired. Lag of that sort wouldn't be too bad of an issue considering it'd be to maintain a level of security and privacy that is difficult to match.( Since Signal is still working on not needing to use phone numbers, to add a privacy/ anonymous benefit to it)
I know both Jitsi Meet and Google Duo( now google Meet) have a bit of info out there about the challenges they encountered upon finding methods to implement multiple-person E2EE video chat.
Duo had a cap of about a dozen give or take, callers at first before they could scale up. Jitsu added E2EE as a toggleable option after getting their initial product out- so this might be the sort of thing where it doesn't start out with lots of Participants, at the beginning.
Finally, you mentioned exploring some of these options not over TOR. That might be beneficial for those seeking anonymity, since TOR usage has been used to fingerprint people when no one else on a network was using it. ( VPN's and Bridges are other methods of course to plug that hole, but usually have to come from the user side at cost and some technical familiarity.)
Push to talk is not something I've seen before- I take it the technical load would be less in that format?