Whether you use a hardware key, or a password manager, in both cases the attacker needs both authentication factors: your password, and the hardware key or password manager.

With a hardware key you gain the advantage of the attackers having to physically gain access to that key.