Here's an excellent talk that addresses your question: Open Source is Insufficient to Solve Trust Problems in Hardware [1].

[1] https://www.youtube.com/watch?v=Hzb37RyagCQ