I tried using the new OpenAI function call API for "code editing", and ran some benchmarks.
Surprisingly, function calls seem to work worse than simple plain text approaches based on markdown code blocks.
I figured others working on AI coding tools might find these learnings interesting or useful, so I wrote up some notes.
I found that having GPT-4 return a JSON dictionary where keys are line numbers and values are their corresponding lines work the best.