"The code itself aims by design to be plain and readable: train.py is a ~300-line boilerplate training loop and model.py a ~300-line GPT model definition, which can optionally load the GPT-2 weights from OpenAI."
Highly educational codebase if you're interested in learning the practical details of training and finetuning your own GPTs.