Visual Anagrams: Generating optical illusions with diffusion models

I’m not sure whether ‘orthogonal transformations’ in this context refers to the usual orthogonal linear transformations (/matrices), but if so then yes.

The man/woman one stuck out to me as well. I probably watched it ten times. Probably because it seems so forlorn.

The mosaics of a duck and a rabbit, however, was hilarious.

That's really cool. Can you do 3x3x3? As in, 9x9 with 81 1-cell cats, 9 9-cell cats and 1 81-cell cat?

That looks more like a cat-aclysm to me TBH. Probably the model was overwhelmed by the conflicting requirements, so that neither the individual images nor the composite image are particularly good. But, as you wrote, maybe they will get better at this eventually...

That is amazing, here's the link for anyone interested (there's a lot of images on that page)

https://dangeng.github.io/visual_anagrams/static/videos/grid...

For me it's the reverse: the color inversions feel hardly more impressive than the morph animations that were all the rage in the 1990, because while I certainly understand how straight-forward color inversion is on the level of pixel data, I still can't "see" that simplicity. It hardly looks any different than an alpha blend with no relation at all.

The rotations on the other hand, wow! It is perfectly visible how the pixels don't change. You can physically rotate the screen and the image "changes". I could not think of a better illustration of how diffusion model images are not just echoes of preexisting images (they certainly are), but solutions to the problem of "find a set of pixels that will match the description of {prompt}". Or in this case, "that will match {A} when oriented this way and {B} when oriented that way".

I can see the woman when the man is shown if I look for it, but not vice versa, for whatever reason.

I missed it, what was infamous about it?

I always thought it was weird that this idea took off with that particular controlnet model. Many other controlnet models when combined with those same images produce excellent and striking results.

The ecosystem around Stable Diffusion in general is so massive.

> Per the code, the technique is based off of DeepFloyd-IF, which is not as easy to run as a Stable Diffusion variant.

I haven't dug in yet, but it _should_ be possible to use their ideas in other diffusion networks? It may be a non-trivial change to the code provided though. Happy to be corrected of course.

Did you mean to say it's _related_? The original "spiral" image by Ugleh is explicitly credited in the "Related Links" section.

You could always make it yourself! Not sure how well the method above would scale up though https://www.createjigsawpuzzles.com/

This research uses DeepFloyd IF, which forbids commercial use. They'd need to find/train another suitable image generator.

Those two are based on previously known ambigraphs:

* very closely https://www.pinterest.com/pin/giraffepenguin--13398215764267...

* or directly inspired by, but the “young lady” prompt triggered the model to pick a dress, and there’s no way to make an eye and an ear or a month and a chocker photo-realistically identical: https://www.reddit.com/r/RedditDayOf/comments/35cjn5/the_cla...

oh hmm, the penguin/giraffe one when I first saw it I was like "that looks like an upside down penguin, where's the giraffe?" Whereas others I immediately saw what it was trying to be.

With that many rearrangeable elements, you could make so many different "valid" solutions, indistinguishable without a photograph, that it would become art rather than a puzzle.

I’d need to check, but if one set of “ear and hole” can be swapped with another set, both sets have to be identical in shape and color. But if they split and attach to other edges rather than swap, that creates further connection.

If you think of the edges as nodes in a connected di-graph of ears and holes, possible pairs are connected: a swap is a two-pair cluster; further connection is a four-element chain with both ends open-ended. If that connection ties to more pairs, you might have a larger cluster of identical hears and holes. Given graph properties, that’s presumably most of them — see the prisoners paradox for why [0].

That would make the puzzle much more challenging to solve if most ears fit in most holes.

[0] The excellent Matt Parker https://www.youtube.com/watch?v=a1DUUnhk3uE but I recommend the following debate with Derek from Veritasium.

The color inversion ones would work well with an E-Ink display.

Most of them are not “illusions” where you perceive two identical segments being different lengths because of tricks of human perception, they are ambigrams. They rely on humans’ ability to think of any three dots as two eyes and a mouth.

They also “copy” the way those networks seem to do so often that they somehow get copyright strikes; they were either prompted on existing solutions or learned them whole through training:

* The penguin and giraffe one is a previously known ambigram, for example.

* The old lady turning into a dress is obviously based on a classic pencil drawing where a similar old lady hiding in her collar turns into a young lady looking behind her shoulder [0]; however, the network interpreted “young lady” and turned into a white dress because color-matching the two different body parts from the pencil outline and turning it photorealistic wouldn’t have been much harder otherwise. There are photorealistic interpretations, though [1].

I’m more impressed by the radically new ones, like the fire flipping into a face—but most of those rely on having two distinct parts of the image be meaningful in their own context, and not relevant otherwise.

The black-and-white inversion man/woman is impressive because the two interpretations are not on separate parts of the image. That’s where you can interpret the quality of the effect as the model having learned how humans perceive and pay attention to dark and light contrasts differently. That one captures an understanding of perception.

[0] https://www.reddit.com/r/RedditDayOf/comments/35cjn5/the_cla...

[1] https://www.jagranjosh.com/general-knowledge/optical-illusio...

some of these style illusions I've seen drawn by hand before, but the lithopane ones are new to me. I'm sure the 3d printing lithopane community will love them

I'm a photographer, and for years I've been pixel peeping at photos taken on phones with "portrait mode"; many years after the first introduction of the feature, regardless of the implementation, results still look crummy to my eye.

Looking at fine elements like hairs (nevermind curly hair) is a disaster, especially when you're used to fine classic german/japanese optics that accurately reproduce every subtle detail of a subject while having extremely aesthetically pleasing sharpness falloff/bokeh.

I've had to swallow the pill though: No one (end users; pros are another story) cares about those details. People just want something that vaguely looks good in the immediate moment, and then it's on to the next thing.

I suspect it'll remain the same for AI generated visuals; a sharp eye will always be able to tell, but it won't really matter for consumption by the masses (where the money is).

well - chuck $10 at it and spend the rest of your month trying other things.

(Back in Disco Diffusion days I was happy to spend money on Colab Pro. It was fun)

I completely disagree. It's fantastic that we can get access to this hardware for so cheap. A used V100 is $1300. You could pay for Colab Pro for 10 years with that, which will get you faster and faster hardware through the years. Where I am, a month is the cost of two bags of chips.

Have you never put quarters into a PacMan machine?

An SGI Onyx has a tiny fraction of the computing power needed to run text-to-image generative models like this.

How do you pay for all that electricity?