I was at a financial software firm that dealt with USB security issues by filling the USB sockets with epoxy. The keyboard and mouse could not be removed from their USB sockets as they were held in place with a metal collar bolted to the case.
Simple and effective, although it destroyed any resale value of the PCs.