Actually you don't need to trust the fab: you can get a batch of identical chips, select X% at random (use dice!), grind off the packaging, examine them under a microscope, and compare what you see to the plans you sent to the fab.

Proving that the plans themselves aren't backdoored (your layout software was written by whom exactly?) and aquiring a microscope that can resolve 32nm features without using any integrated circuts (which, by paranoia, we know will identify images of backdoored ICs and replace them with images of non-backdoored ones) is left as a exercise for the reader.

I'm probably forgetting some other attack vectors here, actually.