An existing implementation of that are Smart Cards. They contain a Public-Private key pair, and they will actually perform the crypto themselves (when plugged into a reader), so the private key never leaves the card.
Here in Portugal our new ID cards (Citizen Card) are already smart cards, so we can log in to e.g. our IRS using it, at least with browsers which support PKCS#11, like Firefox.