Then I have to trust an app (which can have vulnerabilities) or a USB device which can be exchanged for a BAD-USB exploit carrier.
That looks like offloading security issues to the user.
Sounds far fetched? If the code repositories are that valuable, why wouldn't state actors try to mess with the hardware and commit underhanded C or similar?
The repository owners would detect the malicious commit? Well, in that case, why do we need 2FA in the first place?