New FOSS All-Local AI with Ollama, Gemma 2, and Fabric

33 points·MikeBee··6 comments

Seriously smart all-local AI with excellant documentation and clear well-commented code: https://github.com/MikeyBeez/RAGAgent.

View on HN

If you're on macOS, check out the MLX community: https://huggingface.co/mlx-community - Ollama not needed, potentially more efficient(?). The Gemma 2 27B model was broken as of a few days ago, but a fix is already committed and underway to the pip repo. Gemma 2 9B works directly.

Thanks! I've been using gemma 2 9B and Llamma3 8B interchangeably. They're both really good models.

This was not submitted as a link so you can't click on it. That doesn't really follow the HN convention.

PS: Personally I prefer ollama + openwebui <3 For its extensibility. Though I like the playful approach here.

Thanks. I think I figured out how to do this for new posts. The main differences between RAGAGENT and openwebui is that chat history is vector searched for context assemble and fabric is used for system and user messages. It's not just a frontend for the default REPLs. It's a real agent. Moreover, my documentation is designed to allow you to add whatever features you want. It's designed to be an engineer's best friend.

Gemma2 isn't FOSS, it has a rather peculiar license.

That's cool. You can switch the model to whatever you like. type /help and find the command to switch models.

Reply on news.ycombinator.com