If they're serious enough about finding 0 days and exploits to the USB or OS to load this shit any physical access to the box itself is off limits.
Consider what someone with the time, skill, and access to do that could also do without that: opening the case and directly installing some sort of device, planting a camera which records you typing passwords in (“oops, left my cellphone sitting out. Won't happen again!”), planting a radio receiver which opens up all sorts of side channel attacks, installing a passive network tap, etc.
A guard with a metal detector and strict limits on what you can bring into the building or what tools you can use inside is going to do a better job preventing all of those.