A quick Google search implies that these haven’t been proper experiments. It seems as if the brain doesn’t necessarily learn to flip the image, but rather learns to somewhat process the stuff upside-down. A bit like learning to play video games with the controller after using only mouse and keyboard.

The real test as to whether the brain learns to flip the image would be if the goggle-user could read upside-down text after using the goggles for a few weeks, without having read text during that time.