Hacker News Reader

V nice. Can you hook in your own LLM (eg Bloom, t5 etc?)

We’re experimenting with this, and the answer is yes and no.

GPT4 is the only model that can just about run the agent execution, mainly due to context length and quality.

We use our own model for the embedding based code retrieval, and will be replacing some of the GPT3.5 calls with fine tuned models over the coming months.

Would be great if the app allowed connecting with local LLMs like text generation webui. As for quality, it's up to the user to choose their LLM, so I don't see this as relevant.

Also interested. Would be really great to run this against a self hosted LLM agent.

Reply on news.ycombinator.com