Slightly related question: what's a good coding LLM to run on a 4070 12GB card?
Also, do coding LLMs use treesitter to "understand" code?
Also, do coding LLMs use treesitter to "understand" code?
Best models currently: codellama or deepseek coder. 6.7B or 1B depending on how much latency you can tolerate
Treesittee: from looking at the logs of the chat completions requests for Continue or Twinny extensions for VS Code, they both appear to just send a chunk of the document as well as a special placeholder to indicate where the cursor currently is.
If you're wanting something to be your hands so you don't have to type, open-source LLMs and IDE integration is not reliably there yet. Follow the AIDER discord to stay up on the latest in this area.
It's up to the app to put that into the context. Generally, coding LLMs do well if you provide them the source tree, graph, search results, notable files, etc in the context. The is how Sourcegraph's Cody product works, for example.