I believe you can have whatever auth mechanisms you explicitly allow. You will get prompted to use the key if you enable it, but you can also select alternative authentication methods when prompted for the key.