Hacker News Reader

Comment by ansk | Hacker News Reader

Why broadly claim an entire class of generative model outperforms another? They simply show that their particular implementation of a diffusion model attains better performance (for some metric) than the current best implementation of a GAN. Almost inevitably (and likely in the very near future) someone will invest the time and tuning to squeeze out some more performance from a GAN/VAE/flow, and this title will be outdated.

View on HN

Red Sox beat Yankees in the world series =\= Red Sox will always beat the Yankees in every world series game. And I’m guessing most people reading this title know that. I’m sure there are people new to the field or new to CS research in general who might not know how to interpret this, but the only thing most people take out of this title is “hmm, maybe we should consider and study diffusion models more seriously”.

The preeminence of either GANs or diffusion models is an ongoing debate, with evidence trickling in paper by paper. To claim diffusion models beat GANs is like claiming the Red Sox beat the Yankees because they happen to be ahead in the third inning.

It’s like saying the Sox are ahead of the Yankees and it’s the third inning with no predetermined number of innings. That nuance is what makes the title fair.

The Red Sox and Yankees can't play each other in the World Series fyi.

Eh I feel like may be worth cutting some slack here..

It sounds like you’d like the title to be prefaced with “one time that”.

I don’t think anyone reading this says, hey gans are done for. I doubt the authors think that and I (clearly) don’t get the impression that’s what they mean.

Even the first sentence of the abstract:

> We show that diffusion models can achieve image sample quality superior to the current state-of-the-art generative models

Has the word current in front of state of the art generative models.

Why I agree with your sentiment, GANs always seemed finickily and unreliable. I am willing to indulge new approaches to find something more robust

While I tend to agree with you, aren't title like these understood to be short hand for what you described?

Diffusion Models Can Sometimes Beat GANs on Image Synthesis

For an actual example of this more cautious framing, see another paper (also by OpenAI, also comparing two classes of generative models), https://arxiv.org/abs/2011.10650.

It's fine to claim it. It's shorthand for "We achieve smaller FID values (most of the time) than any other generative model."

You're correct about what you're saying, though. It's also worse: FID is a measurement biased towards imagenet-style images, and this model was also trained on imagenet, so it's quite well-suited for achieving small FID values. Whereas for generative anime, it's still unclear what the best model architecture is.

I read your comment, thought “to get clicks, duh”, and then I clicked through the link.

This is a research paper.

I would like to echo your question: why? Does academia incentivize clickbait?

With the rise of Altmetrics? Arguably yes.

IIRC, the last research I saw on this, which was awhile back, argued that clever titles get read more and cited less. https://www.researchtrends.com/issue24-september-2011/headin...

Well, not sure it's any worse than cite-bait?

Are you going on the title? If so, the verb "beat" here admits both a singular past action and a present continuous action.

Reply on news.ycombinator.com