Aider, open source. It's not a vim plugin, but I use it with Neovim.

This particular feature which lets you write instructions/notes as code comments is most relevant https://aider.chat/docs/usage/watch.html