It's kind of like setting up one private/public key pair, then sharing both with everyone in your group and writing each other messages with them. If one of your group become a malicious actor, or if a malicious actor manages to steal the pair, it can compromise the group communications. You also can't cryptographically distinguish between each other since you share one identity.
Mix in video conferencing, though, and you really just have to trust what you see/hear from your peers video feed in addition to trusting your shared key pair isn't compromised.
Ideally, each party member has their unique key pair, they share their public keys between the group, then communicate that way. But you got more overhead in starting a Jitsi meet room rather than just a link and a shared password.