Hacker News Reader

Comment by TheMiddleMan | Hacker News Reader

Another gallery with 80 ddim steps: https://postimg.cc/gallery/b1kn7yd

Thought I'd share this for others interested. I've modified txt2img to save an image after each step. (actually quite easy as you can specify an img_callback to sampler)

Interestingly, both of these runs are using the same seed and prompt, yet they yield different final images, the only difference is the number of ddim sampling steps. I'd love to understand why if anyone has any idea.

View on HN

A couple of replies to https://news.ycombinator.com/item?id=32634807 suggest some sources of non-determinism.

Interesting. I suppose GPUs could calculate things differently. I just checked, I can rerun both 40/80 step runs and the final images are bit-identical to the first runs. So at least in my scenario the same parameters are deterministic, but changing the number of ddim sampling steps changes the result.

Maybe it's doing something fancy with the total number of steps, beyond just stopping after the count is reached.

It will most likely render differently on different hardware since gpu float math is nondeterministic across different environments

How are cloud GPU providers handling this then? Do the fancy A1000 chips solve this?

aaaand there goes people trying to turn this into a 99% compression system

Each sampling step runs at a specific scale, fewer steps would skip some of the intermediate scales

Ah I see, it makes bigger leaps each step to try to get to the same end result in less total steps. That makes sense, assuming I have it right.

Most GPUs are non-deterministic - learned this the hard way in deep learning on pathology data. This is for optimization purposes. In fact, you can set a flag in Pytorch / Cuda to disable this which comes at the cost of performance.

Can you explain? How much does it actually affect results in extreme cases? The source of non-determinism does seem the GPU but parallelism and dynamic allocation in the frameworks. (Also seems that some parts of pytorch still return runtime error if you request a deterministic version). Are there other more performant deterministic DL frameworks?

Do you have a diff/patch of the change to do this?

I may try understanding both StableDiffusion and Python enough to do it, but if you already solved it - that'll be appreciated :)

You can set both the seed and the number of inference steps when running StableDiffusion locally (or in Google CoLab). I assumed that they just set a seed and then generated the image at each inference step. With a decent GPU, it’s only going to take a few minutes.

You could definitely modify it to output at each step, but the output step takes a relatively long time, so it would slow down the process.

Reply on news.ycombinator.com