Yes!, at the cost of recording, speaker stats, and all participants having a pre-shared secret. Or you could use a peer to peer service that doesn't use a media router, if you have few enough participants.
Insertable streams are coming, along with e2ee through media routers. The Jitsi folks are doing great experiments there, and I'm happy they have taken the lead.