Hacker News Reader

Thanks a lot! I plan to fork the project and make a generic version that runs entirely locally using your GPU to do everything. My early tests ran pretty well on NVIDIA 5070. So that's next on my project list to open source in my free time. The only thing more fun that building an AI agent, is using it to build your own ideas!

View on HN

5070Ti user here: We are 150 people in a SME and most of our projects NDA for gov & defense clients absolutely forbid us to use any cloud based IDE tools like GitHub Copilot etc. Would love for this project to provide a BYOK and even Bring Your Own Inference Endpoint. You can still create licensing terms for business clients.

What models do you use that you've found to be powerful enough to be helpful?

I have the same question, do you use already an on prem RAG system?

Reply on news.ycombinator.com