I don't find it super interesting. The orientation distinction is clearly in the training data, and making the algorithm completely rotation invariant would likely a) be more difficult and b) result in worse classification compared to us humans who very much use orientation as a cue having been evolved in an environment with distinct "up" and "down" directions.