At a certain level, if you don't trust someone, you can't do anything.
Let's say you have the source code. You have the source code audited. You have reproducible builds, so you know that the source code is what was used to generate that set of binaries.
How do you know that the platform vendor isn't substituting some other code at runtime? How do you know that the hardware doesn't have a back door? How do you know that the compiler isn't inserting malicious code into the app?
At the end of the day, the only way to prove that something is doing the right thing is to watch what it does. Everything else is an educated guess.
People have built third-party WhatsApp clients, even as Facebook has tried to combat them. What Whatsapp does has been observed to the point where people are able to interact with the server at various points in time. There have been vulnerabilities reported in WhatsApp; no software is perfect, and WhatsApp has made the choice to collect more metadata than, for instance, Signal.
But up until now, nobody has found any evidence that WhatsApp is not end-to-end encrypted as it claims to be. And there are ways for people to find that out, and a lot of incentives for people to do so. Nothing is perfect, and at some point you have to decide to trust someone. I understand that trusting Facebook is fraught. But there are people with good reputations who do good work, like Moxie and tptacek, who do have expertise and can recommend WhatsApp (with an important list of caveats). Whatsapp is not perfect, but most of the options out there are much, much worse.