End-to-end encryption doesn't require participants to receive full-quality video from everyone. Each client can be responsible for encoding their own video feed at multiple quality levels simultaneously – what WebRTC calls simulcasting. That does increase required processing power and upload bandwidth, but not to the point of infeasibility. And you do inevitably leak the identity of the person currently talking, as the server has to know whether to relay the high- or low-quality video stream for each participant to each other participant, and it can trivially tell the difference between the two based on bitrate. But that's much less bad than leaking the whole video stream.