A tutorial on how to fine-tune a GPT3.5 model for Natural Language to SQL tasks and a comparison of its performance vs Retrieval Augmented Generation.

Based on the results, fine-tuning can match and outperform RAG (the approach matches the state of the art on accuracy while being far faster and cheaper). The big challenge for fine-tuning tasks like this is building the training datasets. Things should get even more interesting when GPT-4 is opened for fine-tuning.