On “Learning to Summarize”

I initially thought this would be about "humans" learning to summarize longer texts in a smart way. It's a topic dear to my heart; in fact, I even launched a little side project a few years back, called MNMN [0], with the aim of providing summarization of interesting articles or discussions on the web. I started with my own itch to scratch: Hacker News homepage, which I religiously read pretty much every day.

Here's an example of how the Hacker News home page would be summarized: [1].

Instead, this "learning to summarize" article is about another interesting topic, which is how you teach an AI (GPT-2 and GPT-3) how to summarize text. It's also - bear with me - something dear to my heart, as I used to teach compilers at a CS course in an Italian university in 2004-2006, and I developed an interest for languages in general (not just computer ones).

This one below is the central point of the article, and it is indeed a crucial part of having success with an AI (note: LM stands for Language Model):

> IMO there are two almost unrelated ideas going on in OpenAI’s preference learning work.

> First, the idea of collecting binary preference annotations on LM samples, and (in some way) tuning the LM so its samples are better aligned with the preferences.

> Second, a specific method for tuning the sampling behavior of LMs to maximize an (arbitrary) score function defined over entire samples.

We are, IMHO, at the cusp of a true revolution in linguistics. Can't wait to see what happens in the coming 18-24 months. I expect to be blown away on at least a few fronts.

[0]: https://github.com/simonebrunozzi/MNMN

[1]: https://github.com/simonebrunozzi/MNMN/blob/master/Weekly-Su...

It’s interesting that the title seems like a mix on Learning to Rank, in which a preferred ordering is learned for relevant ranking of search results...but doesn’t mention this anywhere in the article! As noted, you don’t want to do RL if you can avoid it - so I’m wondering if learning decision trees (common in LTR with LambdaMART) can be helpful with summarization to best fit the annotator preferred passages over others. Perhaps the title name is just a coincidence but there is value in exploring, for example, random forests...which are way easier (and faster) than things like RL or CNNs

Wow, that article had nothing to do with the title, for a normal person reading it.

It should really get a much more technical title, befitting the dissertation it leads to.