E2E encryption can be really resource consuming, especially with the amount of traffic used for video chat. You might not want this just because of a worse user experience.
Where people start to feel it is if you want the server in the middle to be decrypting and re-encrypting everything, because it's not that much for one user but it starts to add up for a million users. But the obvious solution there is to use E2EE and then it isn't doing that.