Hacker News Reader

Comment by minimaxir | Hacker News Reader

Note that this technique and its results are unrelated to the infamous "spiral" ControlNet images a couple months back: https://arstechnica.com/information-technology/2023/09/dream...

Per the code, the technique is based off of DeepFloyd-IF, which is not as easy to run as a Stable Diffusion variant.

View on HN

> Per the code, the technique is based off of DeepFloyd-IF, which is not as easy to run as a Stable Diffusion variant.

I haven't dug in yet, but it _should_ be possible to use their ideas in other diffusion networks? It may be a non-trivial change to the code provided though. Happy to be corrected of course.

I suspect the trick only works because DeepFloyd-IF operates in pixel space while other diffusion models operate in the latent space.

> Our method uses DeepFloyd IF, a pixel-based diffusion model. We do not use Stable Diffusion because latent diffusion models cause artifacts in illusions (see our paper for more details).

I always thought it was weird that this idea took off with that particular controlnet model. Many other controlnet models when combined with those same images produce excellent and striking results.

The ecosystem around Stable Diffusion in general is so massive.

Other ControlNet adapters either preserve the high-level shape not enough or preserve it too well, IMO. Canny/Depth ControlNet generations are less of an illusion.

I missed it, what was infamous about it?

It created a backlash because a) it was too popular with AI people hyping "THIS CHANGES EVERYTHING!" and people were posting low-effort transformations to the point that it got saturated and b) non-AI people were "tricked" into thinking it was a clever trick with real art since ControlNet is not ubiquitous outside the AI-sphere, and they got mad.

I rather liked it and actually didn't get to see as many examples as I wanted to.

Is there a good repository anywhere or is it just "wade through twitter"?

not a repository as such but i linked to some good examples in my sept recap

https://www.latent.space/p/sep-2023

https://github.com/swyxio/ai-notes/blob/main/Monthly%20Notes...

It is real art.

Did you mean to say it's _related_? The original "spiral" image by Ugleh is explicitly credited in the "Related Links" section.

It’s a similar topic which is why they credit it but the mechanism is much different.

Reply on news.ycombinator.com