Don't reuse the keys, just generate a different key on each client and add them all to authorized_keys