When the output switches to rabbit the picture actually resembles a rabbit. I am unsure if this experiment was supposed to be a “haha look how stupid AI is” type thing or not, but it seems like the cloud vision api is performing as intended.
The picture is constructed to be ambiguous, and this property is preserved by the rotation: you can still easily see the duck by slightly shifting where you focus.
One mode might be more prominent at some orientation, but the ambiguity is always there so to confidently assign labels and then switch what you assign is an error. So you should be constantly switching as the two classes end up with very similar scores and the noise decides.
Either that, or you decide one label is the most appropriate and then it correctly handle the trivial rotation.
A classifier generally outputs a vector of weights, so it’s likely classifying, say [0.8, 0.75] and then the output is selecting the highest and saying “bunny”. Then you rotate it, and the classifier says [0.75, 0.8] and the output says “duck”.
This is completely reasonable on the part of the network: all things being equal, animals generally appear in certain orientations and we should prefer the interpretation of the amigbuity which respects this alignment, slightly. Example: “bill” down, it looks more like a duck because rabbits rarely have their head in that alignment, while “ears up” it looks more like a rabbit since ducks rarely hold their bills that way.
The problem is actually in how we represent probabilistic information to humans, aka “why the weather man is always wrong”, so it seems like the classifier is randomly flapping when it’s actually perfectly correctly adjusting its distribution of answers based on information we give it.
The original post actually included the predicted probabilities, which are around 80% for duck or rabbit and 0% for the other class. So the neural network really is overconfident.
I can somewhat understand how that happens, but I find it's in an interesting observation (rather than a criticism on the system, though the title is somewhat unclear and had me expect something else).
AI does makes dumb predictions from time to time, but I my opinion, this isn't that strong a case. When it rotate upside down, it does look like a rabbit even to me.
The more interesting 'failure' here to me, is that while the rotation is smooth, the prediction is not, instead it is flickering, which does raise some interesting questions about what the model's internal distribution surface looks like.
This would likely give incorrect or worse results when, eg, classifying ducks and rabbits in wildlife photos — animals come in orientations, and you’ll do a better job classifying them in practice if you respect that.
It’s also not the case that a human would classify it 50/50 in all rotations — it certainly looks more or less like one or the other as you rotate it. Humans are even program,Ed for that compromise: we see faces in one orientation much better than rotated 180 degrees.
I don't think it's meant as an example of a 'dumb prediction'. I think it's an interesting example that inspires (me at least) to think about how this recognition and classification works, and what could cause this effect.
Depending on orientation our first interpretation is also either a duck or a rabbit, because our vision is obviously biased to interpret things in the orientation they are most likely to occur based on our priors that have evolved in the presence of "up" and "down" directions. The AI correctly takes the orientation into account because it also matters in its priors, having been fed training data that captures those human priors.
Now, most computer vision algorithms do strive for some degree of rotation (and translation etc) invariance, because a classifier that gets confused when you rotate the input by 15 degrees or whatever isn't very useful in the real world. But complete rotation invariance would just be a case of Artificial Stupidity in an application that attempts to classify like a human would. Input orientation is meaningful information.