Yep, you move the top of the key away from the door casing, and that's the natural way. Counterclockwise if the lock is on the right, clockwise if it's on the left.
I feel the same way when twist-ties on bread from the store are twisted the “wrong” way (for the 90% of people who are right-handed)
raises left hand defiantly :)
I've encountered an old safe which required two keys to open, one of them turning the "wrong way" - perhaps it's an attempt to slow down a possible burglar by making it "surprising"?