For many people, a core value of a hardware token is "no one ever even momentarily had the key outside of this device, and extracting the key would destroy the device, so the token is the key". If you need a backup you can have a second token with its own key and make both keys authoritative. Or you could have three tokens, again each with their own key, and make it so you need two of the three to get access to the data. If you generate the keys off of the token you can never be 100% sure "no one else has a copy of this key", which should be at least somewhat distressing.