The point is, they are no longer in symbolic memory. At this point they are muscle memory. Quick, which muscle do you use to move your right thumb in? Does it matter?
Another example: There are several passwords that I can type instantly, but I wouldn't be able to recite them.
You rotate it the same direction you're about to rotate the wheel. The vi navigation keys have no such convenient correspondence.