Hacker News Reader

Are y'all running the diffusion in PyTorch eager mode?

AITemplate, stable-fast, or even a torch.compile could get that down to 60ms, I bet. Though I'm not sure the example implementations would work on a non SD architecture.

View on HN

LCMs are actually exactly SD architecture. LCM is initialized from a regular SD unet and finetuned on a new objective. We are already compiling to get to these times. A lot of other people getting sub-100ms times are using fewer inference steps than we do, at a quality tradeoff.

> already compiling

Hmm, well if you mean torch.compile, y'all should still check out stable-fast, which is claiming ~16ms/iter on a 4090, twice that of torch.compile:

https://github.com/chengzeyi/stable-fast#rtx-4090-512x512-ba...

some people even claim they get ~50ms on (powerful) consumer hardware (e.g. RTX 4090).

we still have not been able to replicate those results; also because we want to do it in a distributed way

They might be "cheating" with methods that significantly degrade quality.

Reply on news.ycombinator.com