Not even HMAC-SHA1, just "SHA1"? So you leave yourself open to length extension attacks? http://en.wikipedia.org/wiki/Length_extension_attack
How broken is it here? Who knows. Professional cryptographers would use an AEAD mode with a security proof, or the generic composition of a cipher (w/proof) with ciphertext encrypted by a MAC (w/proof). Instead, this system uses a '70s block mode (that nobody else uses) and a hash function from the '90s.
We are indeed using well-known older algorithms, created in the days when bandwidth and processing power were both a much rarer commodity. This has valuable side-effects for modern-day mobile development, provided one takes care of the known drawbacks.
The weakspots of the said algorithms are also well-known, and have been exploited for decades. We used these algorithms in such a combination that, to our best knowledge, prevents any known attack from possibly succeeding. Although we’d be grateful to see any evidence of the contrary (so far absent from this thread) and update our system accordingly.
While other ways of achieving the same cryptographic goals, undoubtedly, exist, we feel that the present solution is both robust and also sucсeeds at our secondary task of beating unencrypted messengers in terms of delivery time and stability.
[0] – https://core.telegram.org/img/mtproto_encryption.png [1] - http://core.telegram.org/mtproto/description
They appear to be trying to use encrypt and mac, and not as far as I can tell relying on IGE mode to provide authentication. See the "Message Key" and "Important Tests" sections here: https://core.telegram.org/mtproto/description
There is also not, as far as I can tell from reading the protocol description above and the source code here: https://github.com/ex3ndr/telegram-mt/blob/master/src/main/j... (in particular the EncryptedMessage method, lines 447 to 460) any sort of key in the MAC, it's just SHA-1 of the plaintext. It's also not encrypted as far as I can tell, it's listed in the unencrypted "external header" in the protocol description.
I can't think of an attack from this, but the fact that an adversary can generate a valid MAC for _any message_ by simply SHA-1'ing it seems really scary.