So much negativity on this thread. I think it's at least worth mentioning that the paper does present many genuinely interesting ideas; for instance:

1. The change history of a repository contains more information than just the final state of the code. The change log constitutes useful training data which can allow a model to learn things it might not get from just being trained on completed code.

2. Software engineering consists of a lot more than just writing code. They've been able to train a model to do other tasks, such as reviewing code and responding to review comments.

3. Operations such as renaming a variable can be modeled as a single high-level action, rather than as a set of raw code edits. A model trained to generate such higher-level actions can potentially achieve better results. "For example, a rename might touch a file in dozens of places, but a model can predict a single rename action."

4. A developer's recent activity can be used to help predict next steps. They give the example that if you add a parameter to a function and then move the cursor into the function documentation, the model can predict that your next step is to document the new parameter. "Without history, the model wouldn’t know whether the missing function parameter is intentional (because the developer is in the process of a longer edit to remove it) or accidental (in which case the model should re-add it to fix the problem)."

A longer example:

> ...we started with a blank file and asked the model to successively predict what edits would come next until it had written a full code file. The astonishing part is that the model developed code in a step-by-step way that would seem natural to a developer: It started by first creating a fully working skeleton with imports, flags, and a basic main function. It then incrementally added new functionality, like reading from a file and writing results, and added functionality to filter out some lines based on a user-provided regular expression, which required changes across the file, like adding new flags.

This ability to emulate a natural sequence of authorship steps to create a new code file seems similar to "reason step by step", but specialized for coding.