I wouldn't be surprised if GPT-4 is not too good at visual patterns, given that it's trained on text.

Look at the actual prompt in figure 2. I doubt humans would get a 91% score on that.