Slightly related: there’s a YouTube video [1] of a guy teaching himself how to ride a bicycle “in reverse”, i.e. you turn left, and it goes right. It takes some practice but in the end it just “clicks”. So now he’s balancing and moving completely opposite to perception.
So you’re right about the little man. I struggle to see why some arbitrary axis transform is “too hard”.