"Secure" is a relative term. ;-)
The location doesn't need to be "secure" in the military sense of the term. Mainly you want it "secure" in the sense that if you lose the one key, you won't have lost the other key at the same time.
In the end, the two keys are equivalent from a risk profile if they are compromised, and for most people, the one they carry with them is going to be the weak link in the chain, so additional security on the other key isn't really accomplishing much.
In truth, if your home isn't secured, there's enough holes in most people's online security that someone in your home could find a way to compromise much of your online security (and from there, most of the rest).
Just keeping the other key tucked away in a drawer has some risk, but is pretty decent for most people. Keeping it locked in a fireproof safe in their home along with other important documents/keys/bits is a better choice and not terribly inconvenient.
If you want to be more thorough, you can always have a third key that you keep locked away in a bank vault, and you do "wing it" with just two keys for a period of time on new accounts.
What you don't want to do is "wing it" with one key (though AWS seems to think that's the way to do things). That can lead to disaster. What you also don't want to do is only use it for a subset of your accounts, because in practice most people are terrible at risk compartmentalization, so invariably a compromise of some of your accounts quickly becomes a compromise in all of them.
In my experience #1 is the right way for most people to go. There are disaster scenarios where you lose both keys, but usually that's a disaster severe enough that recovering your online access is the least of your worries, and it does a great job of mitigating the much larger risk of mismanaging the security of accounts that aren't secured with the key.