If I'm eating a sausage, I like to be certain that no asbestos was used in its production.
Software security assessments exist for this very purpose. You may personally lack the rigor to do this at home but those who have rigorous security processes absolutely do implement security reviews.
There is a whole industry of professionals who do this work.