StarCoder2-Instruct: Transparent Self-Alignment for Code Generation
github.com
github.com
Is it possible to avoid fine-tuning with RAG for custom libraries. Let us say we have proprietary code libraries and REST API and SOAP WEBSERVICES in the enterprise and I want to use LLM to get code samples that are syntactically correct and usable. If I were to use dump all my proprietory code into a Vector store and use embeddings , will the LLM produce correct code based on a prompt?
1. Big LLMs are good at reasoning. Given a word problem, they can turn it into arithmetic. So they can generate pseudocode for any problem, and with some prompt hints (ex: RAG), translate that to specific API calls.
They likely struggle at 'running' the arithmetic. So they are not perfect at reasoning, as soon as things need to be precise. When reasoning is really a form of translation or summarization, they generally do better, and asking them to 'show their work' in something like arithmetic can nudge them there. I like to think of them as creative Term Rewriting Systems, a useful logic / programming languages family of things. So if you can get a type error on your API call, it can rewrite that portion.
2. They need outside knowledge hints because they don't precisely store data. RAG of APIs solves that.
Instead, they are built on ideas like vectors that add up in n-dimensional space ('king + woman = queen'). They don't precisely know the meaning of an API call, or even save it: imagine lossy jpg compression, you get visual errors when you make it full size again.
So API docs need to be outside the LLM in most cases. In large LLMs, certain APIs are trained on so much - Python data science ones for examples - that the error rate is surprisingly low. Imagine your jpg compression being optimized for cartoons. However errors are still lurking
---
So, the happy case is an API in a language your LLM has been trained on, and context given in a way the LLM can generate code as if it is a summarization or translation problem. An interactive RAG & agent system can significantly further help raise the score because LLMs can reason across attempts in terms of getting new knowledge and correcting missteps. See LLM leaderboards that have increased scores when using LATS.
----
We build this into Louie.AI for analytics (DB calls, Python libraries, visualization tools, team workflows, ...): it's work, but amazing!
At a glance, this looks like a model pretrained to perform prompt-engineering. It should automatically use Chain-of-Thought in its responses in order to improve it's programming abilities, and, therefore be better aligned with users expectations.
It also has reflection. So they include code to execute the model output and return the response to the model for feedback.