How to Store an SSH Key on a Yubikey

You're not wrong, but this is exactly the use case for a USB security stick. The key is in there, cannot be extracted in any way*, can only be "used" (not accessed or copied, just used for crypto operations) while the stick is plugged in, and without it it's impossible to proceed.

It kind of goes without saying that losing the key results in you getting locked out - if there was any other way there wouldn't really be much of a point to the complication of making yourself dependent on a stick.

As a backup, you either have some kind of spare keys in safe storage or reliable access to someone who can restore your access after having identified you.

* in some case you could generate the key beforehand on a computer, and then load it on a stick (unsure about yubikeys though). You should still revoke your key anyway once your stick is lost - as you should assume it could be found and used, sometimes needing only a touch operation rather than a PIN.

That's a very good question and in my opinion the biggest flaw with almost all new 2FA/passwordless systems. The "best practice" is to get a second yubikey or store recovery codes on paper, but that requires keeping them close to you because you'll need to update your backup on every new signup.

That makes these systems entirely useless at protecting against house fires, signups while away from home, or simply services that are too lazy to support your backup style (looking at you, AWS).

I think what we need is one master key that can be backed up in a offsite location (e.g. safe deposit box, lawyer, parents, trusted friends), and then have all subsequent secrets generated from it, or encrypted with it and stored somewhere publicly accessible.

As someone who uses Yubikey for about 5 years for SSH, GPG and O2F, an extra key is indeed the solution I use.

Effectively it means all integrations must support multiple keys, and you’ll have to register both. Of course, this doesn’t work everywhere, such as AWS. In those cases, I typically use my “main” key.

I’d argue that the key breaking due to wear or being lost is less of a risk than human error: just last week I had to enter the admin GPG code for the first time in years, and I forgot it initially, which caused the device to lock itself, and apparently the recovery code didn’t work. Caused me a few hours of stress to get it resolved.

So yeah it requires some discipline. What’s important is that you need to identity the services you absolutely cannot lose access to: in my case it’s my email and password manager. For those two, I properly manage backup codes, regularly test both yubikeys, etc. The rest can all be recovered through email, if it needs to be.

I think the official recommendation is to store a second yubikey in a safe location.

Personally I just generated my key offline (on a tails livecd) and backed it up to two different LUKS-encrypted USB sticks. One of those is stored at my place and another one at a trusted person, in case my flat burns down or so. The yubikey itself only stores subkeys, my master key stays on said USB sticks.

Been using this setup for about 5 years now and it's been working well for me so far. Once a year, I extend my gpg keys expiration time by using on of the USB sticks.

Thinking carefully about loss scenarios is really important, so you're thinking about the right things. I used to have my SSH key on a Yubikey for awhile and now I have it in my MacBook's secure enclave guarded by TouchID. In both of these cases I'm not terribly worried about loss because it's almost always the case that loss of SSH access is pretty easy to recover from. In most cases other people on my team also have SSH access and can swap out my key if I lose the Yubikey. In a number of other cases the cloud server doesn't have any/much in the way of irretrievable data. It's just running various software that is usually version controlled with git. So even if no other team member had access it would be pretty easy to spin up another machine to replace the one that I can't log in to, change the domain name to point to the new machine, and terminate the old one.

You're not overthinking it at all. I have two YubiKeys for that reason. I also have a fire safe with some important docs in it (hopefully the yubikey won't melt!). Plus, I'm older, and I'm getting forgetful, and I worry I'll forget my long lastpass master password some day, let alone all my SSH keys. There are real assets tied to my passwords. Fortunately at my age I have a relationship with my estate lawyer, and financial advisor, who has power of attorney so most of that is safe, but I don't think anyone has really figured out how to handle long-term digital assets. In fact, I'm actually working on a side project to do this.

I don't have two keys. I do keep a fallback method like restore keys written on a safe place.

I did find out however that my old Ledger X can do U2F and GPG so I'm trying to learn how to use it. This Ledger was my first first hardware wallet but I got a coldcard which in my opinion is a better bitcoin wallet. I had no idea what to do with the ledger - second hand market for these things are a big nope. I was pleasantly surprised when I found out there are apps that allow the Ledger to perform some of the Yubikey functions.

> but don't really know where I would feel comfortable storing the extra key

So this doesn't really matter, because with enough incorrect PIN attempts the YubiKey will wipe the credentials. Store it somewhere it won't get destroyed, but don't worry too much about theft.

I thought about this when getting my two YubiKeys as well. I have one on my keychain. Sure I can still lose my keys, but I've had that happen probably once in my life so far. The other one is stored in a relatively safe space in my house. It isn't particularly well hidden or anything, just in a spot out of sight that I can easily remember. After a few months of using them it hasn't been as much of a hassle as I originally thought it might be.

I think that might be only half of the solution. Whenever I have to keep two things in sync—like, in this case, two security keys—, it's always been trouble waiting to happen.

What I'd love to see would be some kind of "trust relationship" between two security keys – ie, if I had set up an account with key 1, but tried to access it with (transitively-trusted) key 2, I would be granted access.

I personally store a printed copy of an emergency-only SSH key (ed25519) in a safe, also formatted as a QR code to quickly import it if something goes wrong. The key is short enough to fit within a QR code length limit, and I can quickly scan it to my computer using my 2D barcode scanner.

I just import my `authorized_keys` that I keep up-to-date on my new server when I do the initial setup.

I was thinking the same initially. But I've been using my Yubikey for 6 or 7 years now, haven't lost it, haven't lost the recovery key which you create during initial setup, and it survived three times being machine washed. I don't know your full situation, but I'd say people are generally much better at not losing things than they think they are.

Encrypt the private key with a strong one time password which you print off (multiple copies) and put in safe places.

This is perhaps not ideal perfect security, but ultimately security is a always a tradeoff with usability and losing a physical item probably has greater chances than someone deliberately trying to get your private key.

You can generate keys off-yubikey and store them in a safe encrypted backup somewhere offline/online/wherever. Then just import those keys into yubikey and use that.

Ideally though you would rotate keys if you lost your yubikey because you never know if the password was compromised or not.

After buiyng a Yubikey and using it for like one year, I now use 1Password to manage all my MFA.

It can work like Google Authenticator (using OTP), except it's not tied to a single device.

Not as safe as Yubikey, but likely a good compromise between security and user stupidity protection.

Also:

- 1Pw blocks itself after a few min without use.

- Installing 1Pw for the first time in a device is a bit more bureaucratic than a simple login.

I really think any MFA needs something to protect users from forgetting stuff to be successful. Even if it's something stupid like going physically to a store. I can't stand the idea that by loosing my two keys I will loose access to all my things and there's no one who can help me.

Most providers also let you create recovery codes. Or signup for TOTP which gives you a QR code.

Print those out and store them somewhere. Then if you loose your yubikey you recover access with those.

If you're too lazy to print them, the just store them in an encrypted tarball, using a very long password (and then rarely open it).

Having the extra key is also fine, but it means you need to have the key around whenever you signup for a new service.

In an ideal world, I would encrypt recovery codes with a public-key and have the private key for decoding them buried in the back yard.

My Precursor† arrived last week, I'm optimistic about the direction the project is going in. Once the ecosystem is fleshed out, we should be able to store secrets with high reliability and deniability, combine this with something like tarsnap and it should be possible to recover everything off a piece of paper, from anywhere, with a fresh device.

†: https://www.bunniestudios.com/blog/?p=5921

If your whole life is on your phone like me, just attach a yubi to your actual keychain (which is stuck to your phone) and then you'll always be thinking about where your phone/wallet/keys/security are all the time.

Then put the second key on your desk. My yubi happens to be annoying to fiddle with since it's on my keychain, so I use the desk one for signing most of the time.

If there's a house fire, you'll have your phone on you. If you lose your phone, you'll have a key at home.

As others pointed out, you just add another factor (TOTP) or print backup codes.

Or just have another way to sign in. Generate random password for root user, store it in your password manager, disable root login via SSH and... whenever you find your user locked out, log via root via console. In a corporate environment, you could audit and alert whenever root logs in as it should be only "recovery" user.

Keeping an extra key stored away works well enough.

IF my primary key gets lots, I'd hope the one backup one won't suddenly fail that same day -- if it does, then there's a problem.

The annoying bit is having to add all 2FA tokens to the backup one -- which is tricky if you want to store if off-site, since you need to bring it in every once in a while to add all the new 2FA secrets to it.

Am I living too dangerously? I keep two U2F keys enrolled, one as a backup in a safe place, the other on my keyring. And for sites that still use passwords, in-browser password manager works well for me.

What kind of key-breaking do you think can still screw me over?

That’s what makes these keys so secure. Most people I know keep one on their person (keychain), one in a locations they have easy access to (hidden at home), and one off-site. You still have a password as the first line of defense so don’t overthink it.

Any system with a backdoor can be exploited. Keep a large number of spare keys that are revokable instead.

Given my "threat model", I don't need a lot of security, so that's one on my person, two other places that are in tamper evident containers.

My thinking is the same. That's why I, in addition to the second USB stick, also have a plain-text copy stored on paper in a safe. EC keys are even short enough that you don't have to involve a printer in the process.

I haven't kept track of these, but are there no solutions to have a master key that one can create any number of keys from? And where that would not require one to update all hundred services using it.

You obviously keep backups (either in form of another YubiKey or backup codes). That's just common sense, you don't only have a single house key either right?

I keep my backup key in a drawer with a bunch of papers and records and shit. It's also where I keep my 2FA backup codes, and a few other decryption keys.

Out of interest if you lost both, what would you lose access to permanently?

My Yubikeys (main and spare) are attached to keychains with AirTags....

> older debian stable

A bit pedantic, but what's currently "Debian stable", Debian 11, has OpenSSH 8.4. The previous release, Debian 10, has OpenSSH 7.9, but it is no longer called stable (instead, it's sometimes called "oldstable").

> which exposes a regular RSA key

With newer Yubikeys you can also use ECC PGP keys:

> Support for Elliptic Curve Cryptographic Algorithms have been added to the YubiKey 5.2.3 and above firmware.

https://developers.yubico.com/PGP/YubiKey_5.2.3_Enhancements...

Another solution for windows: The support is already there within 8.9.1.0 beta release: https://github.com/PowerShell/Win32-OpenSSH/releases

You must then use the SSH installed within 'C:\Program Files\OpenSSH\ssh.exe' and not the builtin within system32.

Not to mention all the network appliances in the world.

While this only works for a subset I'll stay with my RSA key in PIV mode on yubikey. Works great.

I am using this setup for a while now and would like to tell everyone about an advantage in contrast to the 'resident key': You can push a single, identical key to two YubiKeys, making it easier to recover.

Resident keys are (partially?) created on the hardware token and thus can't be replicated. The GPG keys can be pushed to a couple of YubiKeys before you delete them forever (or keep a paper backup somewhere safe).

How does that work, do you know a good tutorial for this?

But PIV only supports keys up to 2048 bits. :<

this.

I suggest testing it yourself in any case, I don't think this article is correct in this. I did try it in the past with Google Titan and not a Yubikey and I could be wrong.

EDIT: `-O resident` might be what is doing it though, I wasn't aware of this option.

If you use SSH ControlMaster sockets, it's not so bad in practice. It can be a pain at times, but I think the security is worth it.

Have you tried using the "export" feature?

Author of the article here. From what I understand it puts the private key material on the Yubikey itself and then during the signing part of SSH authentication the SSH client asks the yubikey to do the signature. The private key never leaves the device.

No, the private key never leaves the YubiKey. SSH and GPG talk to the YubiKey as if it were a smartcard - data is send to the key to be signed or encrypted.

In the blogpost they give an example of exactly that

   $ ssh-add -L
   sk-ssh-ed25519@openssh.com AAAAGnNrLXNzaC1lZDI1NTE5QG9wZW5zc2guY29tAAAAIKgGePSwpBuHUhrFCRLch9Usqi7L0fKtgTRnh6F/R+ruAAAABHNzaDo= cadey@shachi

Seems like the key is exposed as ssh agent.

You don't need this for ecdsa keys, normal ssh agent works just fine as in `ssh -A`.

The feature causing this is -O resident which tells the device, "Hey, you need to remember these credentials" (ie they are resident on the device).

For WebAuthn this enables "usernameless" login. You rock up to a random PC anywhere in the world, go to example.com, just click "Sign in", and your authenticator is like, "Hi example.com, according to my records I am archi42, user 123456-ACBDE-123 and as proof here's a signature made with my unique private key" and the site checks its database and signs you in. Convenient and fairly secure (most devices with such a feature expect a PIN, or a fingerprint, or some such factor beyond "something you have" in the form of the authenticator itself).

For SSH, this means the magic file that makes SSH with FIDO work can be regenerated on another client machine by just asking it to spit out the credentials.

Chances are your device does not have this feature, usernameless login on the web is rare, so few people need this, and of course it's a considerable extra hardware implementation burden. Yubico products have it though, as do some others, and the phone implementations (iPhone, newer Android) likewise.

If you mostly use the same machines (laptop, maybe a desktop) for SSH, the resident feature isn't important, just don't write "-O resident" and remember that although they aren't a security feature the resulting files are unique and if you don't have them you can't log in. If you regularly use different machines for SSH login because you're say, a roaming technician logging in to physical hardware on site or you insist on travelling very light, then it's very valuable and worth upgrading to get the resident feature.

The support is substantially more than "including them in a list of recognised key types" because the FIDO device is deliberately unwilling to do anything except emit FIDO-style signatures, so you need to understand those signatures and verify them.

On the other hand, probably we should have learned by now that even apparently trivial verification steps are too easy to get wrong (or plain omit) and so you really want to delegate all of this work to just one implementation which was actually written by people who know what they're doing and, preferably also formally verified as correct since people who "know what they're doing" still make far too many mistakes. Thus, maybe there shouldn't be so many independent (and likely in some cases, wrong) implementations of this check.

I dated a charismatic christian who did the whole laying-hands on thing and speaking in tongues as a kid. Weird stuff. But not any weirder than any other religion system comparitavely.

As someone who was in an Awana...club(?) for most of my childhood, I never had this happen. That sucks. Some people are way too crazy.

Doing that means I have to use GPG. I am not paid enough to use GPG.

I guess it's to clear the confusion of, for example, what date would "04 07 2022" be, is that 4th of July or April 7th?

"M05 27 2022" seems to indicate that it's in fact MM DD YYYY not DD MM YYYY

M for month?

I'd expect the keywords "linux PAM" would be your friend when looking for this. https://wiki.archlinux.org/title/PAM