You can send a single stream with E2E, just as your can send a PGP message to multiple recipients. Each packet/bundle/whatever has the decrypt key encrypted against each of the recipient keys.
[0] https://superuser.com/questions/554513/pgp-encrypt-single-me...