I think that the difference is that someone can have a similar experience to Gmail by using their own mail server and any mail client. The same cannot be said about Uber, for example.
You can't call a friend and ask them for a lift?
>But really isn't it just us trusting Gmail not to pull our access
Nope, not trusting Google and I do not use it (or any other cloudy stuff) to keep what is mine.