Is this a self-hosted GPT model? One of the smaller models? Fine tuned on Crunchbase data? Any insights into how this was put together?
The article describes how the prompt itself is constructed - to include details of the database schema plus some example data rows, followed by the user's question.
The prompt is then sent to the GPT-3 API. The results are then sent through the API a few more times with some extra prompt engineering before deriving the final result by running the SQL against the original database.
Edit: I noticed you edited your comment above with some details. It seems that he used GPT-3 just to create the SQL statement which was then executed locally. So GPT-3 never had the actual data set, just a prompt that included some details of the schema.
As a result I think any sort of "success" here is fairly limited when GPT-3 is blind to the actual data and has limited knowledge of the schema to be able to formulate queries that imply some knowledge of the data.
Training these models with extra data turns out to be incredibly expensive and relatively ineffective.
Instead, the most interesting research is all around tricks like this - figuring out ways to round-trip to the language model, then query other sources of data for the information that it needs, then sending more prompts to the language model again.
I wrote a tutorial about a pattern for doing that a couple of weeks ago, but this SQL trick is a lot more sophisticated than what I've done so far: https://simonwillison.net/2023/Jan/13/semantic-search-answer...
I can see that it's expensive, but have you tried it for effectiveness?
BTW, your approach is very cool here.
I've seen a few other people suggest that time tuning GPT is unlikely to give better results than just feeding the regular model a few examples in a regular prompt.
I've yet to see anyone talking about a GPT3 fine tuning project that went really for them. Maybe I haven't looked in the right places.