If all you have is the URL, then the server sees the encryption key.
Video conferencing also rarely has users register. So there isn't a way to validate users either. And even if they did register and users didn't care about the extra friction, multiple devices means either the server stores your private key, or you have many keys which is much harder to verify.
E2EE is much easier on phones, which is why Signal is so good. The identity is your phone number, and you can only have one key associate with your number. That key never leaves your device. Conceptually easy.
Video conferencing has none of those advantages, and I don't know how you would make it conceptually easy for users without reducing the security.