That is weird. It decides to flip a column by summing up the vector distances between the pixels in that column vs. the pixels in the previous column. I'm wondering if that since there is so much gray in the painting, this causes a large accumulation of error in gray vs. slightly different gray which outweighs the orange vs gray.

I do know that euclidean distance in RGB space is not that great a metric for perceptual similarity. Perhaps everything should be done in YUV instead? I will try tonight. Thanks.