Yes, because the key is still private. Any other machine to witch you can login with that private key is still off limites.
Personally I think one key per client is a good way when not using a hardware security module (e.g. yubikey) as the public key then identifies a unique client machine (e.g. your work laptop). This would help identify which client was breached in an eventual attack. I do think however that I would prefer the Kryptonite solution or using a Yubikey going forward.