Can someone explain to me the "Model Formats" section at the end of that page. It makes sense to me as a description of how to format the training data, but it also says "Use these templates to explore the model’s capacities"
How would you make use of this prefix format in a prompt?
<reponame>REPONAME<filename>FILENAME<gh_stars>STARS
code<|endoftext|>Page 30 of the TR has a few examples:
https://drive.google.com/file/d/1cN-b9GnWtHzQRoE7M7gAEyivY0k...
Note that they were produced with StarCoderBase. E.g., here is one:
Input:
<commit_before>def fibonacci(n):<commit_msg>add type hints to function<commit_after>def
fibonacci(n: int) -> int:
The <reponame> token specifies the name of the repository, and the same goes for the filename. A high gh_stars count can make it mimick popular open-source libraries. Admittedly, there is a bit of information that can be retrieved from those tokens, but not as much as the others.
The more interesting ones are the commit tokens, which you can use to ask it as an assistant to implement a commit whose high-level description you give in plain English, and the issues token, to make it solve an issue.
The Fill-in-the-middle tokens are most useful for autocompletion: it will allow it to gain information from the code that is present after your cursor, instead of just the code before your cursor.
Of course, in practice, those tokens are meant for code editor plugin writers. Normal users won’t know about them.