Will the solution to this, then, be to have some sort of "smart card enabled device"? For example, assuming TOFU, you manually accept all device's public keys (and all devices, including cables and stuff will have one of these). Then, the computer will have to verify all actions done by those devices by sending a challenge for each action. But this seems impractical and inefficient...
Perhaps physical security is the only way for this...