I'm not associated with Signal or Moxie (though I've been a silent fanboy for ~15 years-- [[hey Moxie if you're reading this and still are hiring, ping me - contact info is in my profile]]), but I'd inherently trust the application more than an average application or company because:
a) Moxie has a track history of having a lot of personal integrity with regards to security. Some might say this is blasphemous but I'd put him up there with Bruce Schieder.
b) The whole source is available on GitHub, compile the APK and see if it matches.
c) it's incredibly easy to take an apk and disassemble it, to see if there are backdoors to begin with if you are really that cynical.
Don't get me wrong, I'm a tinfoil hatter. I use GPG, run my own MTA for anything even remotely important, use DDG over Google, donate to the EFF and use their HTTPS Everywhere plugin, have all of the Ad-Opt outs enabled that Google/Doubleclick/etc make available but try to obscure, etc. I'd be willing to bet that Google is collecting way more information than Signal is.
But hey, that's why rev-eng is so important. A wiser man than me once said "Don't turn it on, take it apart" ;)