There have been a number of independent security audits over the years, which are easy to find. WhatsApp code was never open, even pre-Facebook.
That said, I (like others) am a bit concerned about the lack of updates to Signal-Server (https://github.com/signalapp/Signal-Server/commits/master). Commits seemed to suddenly stop April last year, and I'd be very surprised if the actual Signal Server that's running in production hasn't been updated over such a long period. Would be very happy to be proven wrong here, or to be pointed in the direction of anything that might explain the lack of activity.
On the client side I find their interest on self-updating problematic (since they may silently push updates to specific users), but at least you do have the option to remove it.