I can't imagine switching keyboard layouts 3 times, your mind must be extremely nimble. Currently, I have a picture of the dvorak keyboard pinned as my desktop wallpaper so if I forget location of a key I can look at desktop quickly.
But did the nimbleness allow him to switch so much, or did switching so much cause the nimbleness?
I can juggle 5 balls, and when I do, I often hear people say something like 'wow, you have such great coordination, I could never do that', but they don't realise that before I started practising with 5 balls I didn't have the coordination either! I would throw 5 balls up and 5 would hit the ground.
Practice and determination over time built up the necessary coordination and skills to allow me to do it now with relative ease.
The same principle applies to learning keyboard layouts and almost everything else.
I guess the one thing that does help is memory. I was able to ditch the printed layout after a couple of days. But even then, it was slow going for the first week or so.