That's a very cool idea, with the caveat that if an attacker knew you were doing that, the search space for your key would shrink.