> Let me put it this way - name a couple of open design issues with the current Telegram protocol.
https://core.telegram.org/img/mtproto_encryption1.png
The use of encrypt and MAC is the one that should jump out at even a crypto neophyte.
There are some nice proofs around this[1] that I've read before. The important one is that encrypt then MAC guarantees INT-CTXT: it's computationally infeasible to produce a ciphertext not before sent by the sender. INT-CTXT implies INT-PTXT which is the weaker claim but typically the one people associate with the function of a MAC: that you can't forge a plaintext the sender never sent.
While E&M isn't immediately bad it has been used as an avenue for attack[2]. If Telegram's aes_ige_decrypt() function accidentally overflows a buffer... and because they're doing E&M it must call aes_ige_decrypt() on whatever message I decide to send it.
Now, Telegram can claim that those attacks don't apply because of their use of IGE, and they may very well be correct! But then we move into IGE: there are known attacks that show chosen error introductions can cause the stream to resynchronize without error[3]. What does that mean in practice? Who knows: we know IGE is broken, but we don't know how badly because no one actually studies IGE, or knows how it is implemented in this system!
The real failings here: Telegram uses a MAC with less security guarantees AND a cipher mode that would be charitably described as anonymous, instead of using a provably secure MAC along with well-studied modes.
Now, I'm a big security dummy (seriously) but if there's one thing people far smarter than me have banged in my head, it's this: you follow the well-beaten path precisely because it is well-beaten. When you start venturing off the trail into the woods of funky block cipher modes and known-problematic MAC modes, your margins get a lot thinner.
[1]: http://cseweb.ucsd.edu/~mihir/papers/oem.pdf
[2]: http://www.thoughtcrime.org/blog/the-cryptographic-doom-prin...
[3]: https://groups.google.com/forum/#!topic/sci.crypt/4bkzm_n7UG...