Oscar, an open-source contributor agent architecture
go.googlesource.com
go.googlesource.com
That’s one reason I added interactive help [0] to aider. Users can type `/help <question>` in the app to get AI help, based on all of aider’s docs. I’ve been considering turning it into a bot for GitHub issues and discord.
Dosu [1] is another app in this “triage issues” space that looks really good when I encounter it in the wild.
No need to use some elaborate AI solution for such a simple (and long-time solved) problem.
Aider can tailor the help response to exactly their question and situation. It also always links to relevant docs in these help responses.
I've pasted doc links in reply to user questions in discord, and I've pasted /help output. The latter is far more helpful, and includes the links as a bonus.
The moment your question is more than 5 words long you can forget about it.
In the case where someone asked your exact precise question including follow-ups, you get the same (or better) results from search.
In most cases for a small project, no one has answered your precise question, so you might need to read docs and figure it out. It’s often much quicker to use an LLM here (though sometimes less accurate than the hypothetical matching search result).
As a benchmark, try using an LLM vs. search for answering questions about Nix. I find it to be 10-100x more efficient than searching and reading through the docs (including hallucinations and time spent corroborating, since usually I can validate the answer by running some command). This is perhaps a cherry-pick for LLMs, but it should illustrate clearly why “just use search” is missing the value prop completely.
Here's an example of it in action: https://github.com/LibreTranslate/LibreTranslate/issues/632
It's been working pretty well.
[1] Github bot that does direct contributions https://github.com/tscircuit/bunaider [2] Example contribution https://github.com/tscircuit/checks/pull/8
It is a bit hard to get it to work reliably except for small changes.
I'm curious if there are any projects folks like for the indexing/embedding step, like Git repo -> vector index / graph index of code + comments + docs?
(I am not as interested in the RAG, LLM, UX, etc after)
Such agents would mostly provide guidance to jumping through the hoops necessary to get your PR in a state where it's mergeable according to the project's contribution guidelines — removing the need for humans to be that guide. (In other words, they'd act as a "compiler for your PR" that you could iterate on, with good English-language error messages. IMHO something that should already exist locally — but git has no local reified concept of PRs, so this is hard.)
But I would expect that in almost every case, the project's human maintainers would still eventually step in to review the PR, after the bot seems happy with it.
---
Mind you, in theory, for a particular set of limited-scope (but frequently occurring) problems, Tier 1 Support agents are usually empowered to solve those problems directly for a customer. And likewise, there could potentially be a set of limited-scope contribution types that a Tier 1 Project Contribution Auditor would be able to directly approve. Things like, say, fixing typos in doc comments (gated by the bot determining that the diff increases semantic validity of the text by some weird LLM "parseability" metric.)