Hacker News Reader

Comment by azinman2 | Hacker News Reader

> Is it just me or does anyone else cringe when they read "is/is not all you need" in the title of an AI-related paper?

I believe it's a reference to 'attention is all you need,' which is a very famous paper now. I'm guessing you probably already know that -- did you cringe with it? It was a landmark paper so at least it's worth hyperbole in the title. Maybe I'm missing that its usage started before that.

Reminds me of my ML prof who would complain about people using the word 'optimum' or 'optimized' in titles of papers... one could almost always optimize more, and with respect to what is not specified by the title.

View on HN

Yes, aware of the reference and hence the cringe. 'Attention is all you need' is a clever play on words for a seminal paper whose title directly related to its main scientific contribution.

Not sure that every other AI-related paper needs to copy this format, this trend is the academic equivalent of click-baiting (seemingly trying to associate with the Vaswani et al paper) and in my anecdotal experience the usage of this play on words seems inversely correlated with the paper's quality.

I believe it has an analogue with Heinlein's "there are two kinds of jokes; funny always and funny one time". Subsequent uses are a bit cringe in the same way that a subsequent telling of a funny-one-time joke is.

Those kinds of references/cliches in titles always bug me too. Always feels lazy. I chalk this one up there with:

Dr. Strangelove Titles - "<product>: or how I learned to stop <doing something> and love <idea>"

Dairy Farmers of America reference - "Got <Product>?"

Breakin' 2 reference - "<Product> 2, Electric Boogaloo"

Honorable Mention, titles that end in phrases like "And That's a Good Thing!", "Here's What That Means", "This is Why That Matters"

"Repeated Tropes Considered Harmful"

> 'Attention is all you need' is a clever play on words

Can you elaborate? (I'm not a native speaker)

I think the clever play on words is the fact that the title is cute and references the main finding directly: transformers have a mechanism called 'self-attention' that distinguishes them from previous approaches.

Also likely referencing "Love is all you need", from The Beatles' https://en.wikipedia.org/wiki/All_You_Need_Is_Love

I’m sorry, but what is the play on words in the title “Attention is all you need” here?

"love is all you need" is the Beatles lyric they're referencing, the title could also be read as a statement on the basic human need for attention, like love, so it's a play on words for me. Do you prefer humorous allusion?

It has been a wildly surprising fact over the last few years that attenion basically is all you need, in a sense much stronger than the one intended by the authors.

> Maybe I'm missing that its usage started before that.

Afaik it wasn't used prior to it. While attention (and residual connections) were used with recurrent / auto regressive models, the paper showed that an encoder-decoder (non auto regressive and auto-regressive) architecture with just attention (and residuals) is sufficient to achieve great results given that you provide positional embeddings.

So prior to transformers it was RNN + Attention (+ Residuals), but the paper claimed that Attention (+Residuals+Pos Embeddings) is all you need.

Reply on news.ycombinator.com