My understanding is that the private key is on a CAC card[1,2], which is issued to all DoD employees and contractors who need access to DoD resources. Readers are usually built into the keyboard, and I'm guessing that there's something similar to ssh-agent running to maintain the key until you're AFK.
--