> this is pretty much what Matrix does, if I understand correctly.

I think it has senders encrypt messages with each room member's public key, rather than a single shared key. (At least, that's what the behavior I've seen suggests to me.)

Here's the spec, in case you want to comb through it:

https://spec.matrix.org/v1.11/client-server-api/#end-to-end-...