Sweden got a system a bit like this called BankID.

BankID isn't operated by the government but rather by the system critical banks however it's really ubiquitous today and supported by most government sites, banks and other high security web systems.

BankID comes both as hardware keys with smartcards or as a mobile application where you use a local key to sign a request.