Hacker News Reader

The pace of progress here is pretty amazing. I loved how easy it is to get llamafile up and running, but I missed feature complete chat interfaces, so I built one based off it: https://recurse.chat/.

I still need GPT-4 for some tasks, but in daily usage it's replaced much of ChatGPT usage, especially since I can import all of my ChatGPT chat history. Also curious to learn about what people want to do with local AI.

View on HN

My primary use case would be to feed large internal codebases into an LLM with a much larger context window than what GPT-4 offers. Curious what the best options here are, in terms of model choice, speed, and ideas for prompt engineering

I came across https://github.com/BloopAI/bloop that seems promising, but it's GPT-4 based. It uses Qdrant as vector DB for RAG.

Bloop is amazing. Once you use it you stop building your own DIY codebase QA setups.

Yi-34B-200K might be something to look at.

* https://huggingface.co/01-ai/Yi-34B-200K

What's up with the landing page though? Unless I'm not well awaken, there doesn't seem to be a download section or anything.

Reply on news.ycombinator.com