This is not my understanding of the situation at all. There's no end-to-end encryption by default [0], and the end-to-end encryption they do have received significant controversy at launch [1] for being essentially a "roll your own" crypto solution which indeed ended up being found to have some issues [2].
They disable the OS backup and instead they effectively store all their user's contacts, messages, media, etc. directly on their servers except for the conversations that the users directly opt out of by turning on e2e. They've promised since 2014 to open source everything but the backend, which stores all this data, is still closed source.
For small group or individual messaging, whatsapp, signal, or matrix are far better choices. I think it's worth acknowledging that telegram has a much bigger focus on large groups and therefore has to make different security tradeoffs, so I think if we consider telegram a social media service it's pretty good -- but is not the best messenger.
[0]: https://www.howtogeek.com/710344/psa-telegram-chats-arent-en... [1]: https://www.vice.com/en/article/wnx8nq/why-you-dont-roll-you... [2]: https://eprint.iacr.org/2015/1177.pdf