Not surprised; there were so many variations of BASIC and unless you train ChatGPT on a bunch of code examples and contexts then it can only get so close.
Try a local LLM then train it
Try a local LLM then train it
How do you do this?
1. Gather training data
2. Format it into JSONL or Hugging Face Dataset format
3. Use Axolotl or Hugging Face peft to fine-tune
4. Export model to GGUF or HF format
5. Serve via Ollama
https://adithyask.medium.com/axolotl-is-all-you-need-331d5de...
https://www.philschmid.de/fine-tune-llms-in-2025
https://blog.devgenius.io/complete-guide-to-model-fine-tunin...
When I fine tuned a Mistral 7B model it took hundreds of examples in Alpaca style
It’s a lot of work. Maybe OpenAI has a more efficient way of doing it because in my case I had to manually adjust each prompt
You can add some tutorials/language docs as context without any problem. The bigger your project gets the more context it gets from there. You can also convert apis/documentation to a RAG and expose it as a MCP tool to the LLM.
You mean around 3000 files with 3000 characters? That is a lot. I've played with some other LLMs in Agentic AIs but at work we are using Copilot, and when I add context through drag and drop it seems to be limited to some dozen files.
Gemini is better than Sonnet if you have broad questions that concern a large codebase, the context size seems to help there. People also use subagents for specific purposes to keep each context size manageable, if possible.
On a related note I think the agent metaphor is a bit harmful because it suggests state while the LLM is stateless.