Your typical user is much more at risk with end-to-end encryption than with a well-architected SaaS product, since their likelihood of being a victim of some sort of easily preventable fraud is much more likely than them having their data misappropriated. (Think of how Gmail automatically handles con artists, phishing scams, viruses, XSS attacks, SPF/DKIM/DMARC, etc.)
Because of this, users tend to be better off having their data protected through a combination of technology and legal agreements rather than protected solely through technology, although this can be difficult to communicate.
E.g. how would you communicate to your (archetypal) grandmother that she's better off using Gmail than the same sort of setup as Edward Snowden, and make her feel safe doing so?
Not all e2ee products are created equal. I think your description is accurate for the PGP ecosystem, because for example it's hard to be sure you've got the right key from the key servers, and anyone who can use email can contact you. In fact most crypto folks I know believe email is an unsecurable platform.
However good e2ee systems provide strong Authentication of the data's origin, which is often more valuable than the encryption. In SpiderOak's Semaphor for example, a team admin approves new team members before they can communicate with other people on the team. It's explicitly a tool intended for safe internal communication with a specific in group, such as an enterprise team.