While a part of me agrees, I don't want the Chinese to have access to even non-critical things. What if they insert some kind of small component that can be embedded in a PCB stackup that siphons data from a bus, or introduce a high Q resonator that creates an easily discoverable signature when hit with a certain frequency, or something else (like The Thing - https://en.wikipedia.org/wiki/The_Thing_(listening_device)?
x-ray inspection at the bare-board level would check for both of those things (of course), but usually x-ray inspection is done by the assembler after boards are populated - not before - and I've never heard of it being one on a bare-board as an inspection step post receipt. So something like this actually could be missed.
You wouldn't even need to do it to every board, you could do it to some fraction of them, and have (essentially) a secret frequency vulnerability lying in wait. Basically it would invalidate your stealth/LO (obviously this is dependent on shielding and a lot of other things, but you get the idea).