I'd doubt users of an encryption app not using a simple checksum to verify its legitimacy before using it.
It reminds me of a time when I wanted to get in touch with an HN user... they had their pgp key in their user page... I tried sending the email but it didn't work (formatting issues). Finally I reached this user and asked what was up with it... and he responded by saying, "Oh, noone in the past 10 years has made use of this!" And this is a pretty smart security guy.
I think there are very, very few people who actually take the trouble of verifying checksums and everything. I used to do it /sometimes/ years ago, but it gets to be a hassle pretty quickly and you start taking shortcuts everywhere. A lot of us sort of plan to start doing it, but never actually do it. Kind of like exercising or eating healthy or whatever. :)
Anyway, as for this news about ISIS having their own encrypted chat app - I'm happy to hear it. If they made it on their own, it's bound to have a good dozen holes that NSA will have no trouble poking through. :)
It's very hard to think someone would get to the trouble of creating and using a secure chat application, but then fail to secure its distribution.
I've tested many applications which claim to be secure, designed for security/privacy sensitive tasks, yet are very easy to compromise (simple OWASP top 10 stuff).
Even if the app developers are great and know their stuff, I can still see them slipping up on the distribution. It's normally handled for most developers and is outside the realm of any secure development guidelines they might be following.
If both you and I have an shared app, then we have some shared data and protocols we can use to protect our communications and maintain security, but when you download an app you do not yet have any protection unless that protection is provided by an app store (and such app store protections disappear at the first court order).
Bootstrapping security over insecure channels against nation state threats is close to impossible, maintaining security you have already achieved over a brief window of time is still hard but achieve given existing technologies. I see no evidence ISIS is capable of doing either. As further evidence observe that the US just blew up their super secret cash reserve.