Seeing Faces in Things: A model and dataset for pareidolia

I remember reading somewhere, maybe in an essay by John Updike, that Abstract Expressionists like Jackson Pollock, who aimed to produce purely nonrepresentational paintings, had to be careful that face-like figures did not appear in their works unintentionally. They wanted to create art that had aesthetic value without recognizable images, and the effect they were seeking would be destroyed by an accidental smiley face or two among the vigorous brush strokes and dripped paint.

This sounds a little like it might be related to how adversarial images work, because it sounds like the same kind of idea - you trick an image classifier into believing that it sees something that isn't really there.

In a way, I guess pareidolia is just our version of adversarial images - It's just that we ascribe more obvious things (things that look like eyes, noses, mouths, etc) to the reason why we see faces, whereas I imagine an image classifier just happens to see random pixels that are the same or something like that.

Just the other day I was at the Mille Lacs Indian Museum in Northern Minnesota. It has a really beautiful and well put together feature showing a collection of full sized Wigwams across the seasons.

One of the wigwams for the Winter season had a very large piece of birch bark with a very obvious face in it. It was so obvious that I thought it had to be some sort of Easter egg by the museum.

Pointing it out to my wife however, she couldn't seem to see it. She was like "maybe it looks like a face if I really try". Brain really plays tricks.

This is exactly what CNNs do. Recognize patterns in transferrable areas of images. Once that feature map is generated, successive layers just look for the same patterns. We see patterns in faces, and so does AI if it uses a CNN or CNN-like model.

There's an area of the brain called the fusiform face area which, despite its name, may actually be an area that's involved in visual expertise rather than faces per se: https://en.wikipedia.org/wiki/Fusiform_face_area

This is interesting in that I imagine this is similar to visual expertise rather than faces as such - I presume you could train a model to see areas of images as birds in the same way.

Trying to suggest a serious link between the two is a bit ridiculous - rather like the idea that plants which look like dogs can heal dog bites (which is itself a form of over-recognition!) - but I find the parallel curious.

Back in 2018, I ran a little test to see if I could push Google Cloud Vision to recognize objects, shapes, or patterns in clouds. No matter how I treated the images ahead of time, the answer always came back: clouds.

Would be interesting to see how much free-association and hallucination have "improved" the results with the current generation.

But does the “AI” realise these aren’t real faces?

pareidoilia are a natural side effect of any pattern recognition machine

A large part of the brain is used for face recognition. There are dozens of regions each dedicated to process one feature of the face. The brain is also a generation machine. With only a few features recognized the brain can generate the rest of the face features, thus recognizing it as a face.

With generative AI, it works the same way.

I imagine faces on the fronts of people's heads. I know that this is common. Is this a consensual hallucination?

I must link https://www.reddit.com/r/Pareidolia/ for anyone who likes findings faces in random places. There was another sub about things we see in clouds, can't find it.

I used to be so good at this. Then I started smoking cannabis. Then I became ultra good at this. Stains on walls became unbelievable works of art. I miss those days. I also use to hear symphonies in my brain as I’d fall asleep. Crazy, except it was beautiful

as a potential step up from overly sensitive pattern matching: somewhere I ran across the idea that our close primate relatives enjoy sleight-of-hand magic tricks, but more distant ones do not.

One half of the entire basis of modern machine learning is creating algorithms capable of reaching the pattern recognition levels of humans.

This is a given.

There's a related line of research that concerns computer vision models and optical illusions.

Next series: You should take probiotics for your gut bacteria and so should AI (sponsored by nature.com and Yakult[tm]).

What is even science-worthy about this? If you can see a face in a cartoon drawn with a few lines, then those lines may appear in a cloud, stone, whatever. News at 11.

I wonder how much further along we will get creating human-like intelligences until Occam's razor suggests that the (in evolutionary scale) sudden emergence of human intelligence ~20,000 years ago was the result of the efforts of an intelligent force