I've been using the openai apis to write and edit code for 3-4 months now. So I think that makes me an old timer (ha!). I have also been building tooling for improving the chat based coding experience [0]. All of this work has given me an opportunity to think about how to work best with GPTs on coding, and I've shared some thoughts about this in the past [1].
Here are some of my thoughts on how to code with chatgpt. Many overlap with topics covered in the Nature article.
- It really does help to think of chatgpt as a junior coder. I mentioned this in the writeup about my first AI coding project [2]. A bunch of things in this list would also be helpful when working with a junior dev.
- GPT isn't good at code architecture. It will repeat code and take the shortest, laziest path to coding up your request. You need to walk it through changes like you might with a junior dev. Ask for a refactor to prepare, then ask for the actual change. Spend the time to ask for code quality/structure improvements.
- Don't copy-and-paste between a chat session and your files. Use a tool like Copilot [3] or aider [0] that will give GPT your code to edit and apply its changes automatically. This is critical as it removes friction and lets you iterate quickly on code with GPT.
- GPT is amazing at generating fresh new self-contained code. It takes more skill and better tools to work with it to edit existing code.
- Break down a big change and ask for a series of smaller, self contained steps. This is a smart way to code solo, but really helps GPT succeed at more complex code modifications.
- GPT has a wide breadth of coding knowledge, and impeccable command of syntax... but it is sometimes overconfident about details. So it will usually choose the right library for a task, but sometimes make an error about the details of the api/method/params. Paste doc snippets or error messages into the chat and it will fix those bugs.
- GPT is really strong at churning out good boilerplate and roughing in a solution. This reduces the activation energy required to start larger changes or changes which will require unfamiliar libraries/packages/languages. GPT can can quickly prepare the ground for you to do the interesting work.
- If you can, use gpt-4 (not gpt-3.5-turbo) since it can successfully generate larger, more complicated code changes.
[1] https://news.ycombinator.com/item?id=36020809