Great question! We iterated on the prompt for several days and manually verified the results for ~100 users.

The results were pretty good: https://gist.github.com/gaurav274/506337fa51f4df192de78d1280...

Another interesting aspect was the money spent on LLMs. We could have directly used GPT-4 to generate the "golden" table; however, it's a bit expensive — costing $60 to process the information of 1000 users. To maintain accuracy while reducing costs significantly, we set up an LLM model cascade in the EvaDB query, running GPT-3.5 before GPT-4, leading to a 11x cost reduction ($5.5).

Query 1: https://github.com/pchunduri6/stargazers-reloaded/blob/228e8...

Query 2: https://github.com/pchunduri6/stargazers-reloaded/blob/228e8...