To keep track of your master password, just write it down and put it in your wallet/purse. We're used to and experienced with securing physical objects, and while it still offers an avenue to expose all your passwords, it requires a lot of extra work to do so.
Basically, storing your master password physically only really vulnerable to highly targeted attacks.