How do I use it with ollama models?
1. Install Shimmy:
cargo install shimmy
2. Get GGUF models (same models you'd use with Ollama):
# Download to ./models/ directory
huggingface-cli download microsoft/Phi-3-mini-4k-instruct-gguf --local-dir
./models/
# Or use existing Ollama models from ~/.ollama/models/
3. Start serving:
./shimmy serve
4. Use with any OpenAI-compatible client at http://localhost:11435How do I know for sure it is checking ~/.ollama/models/ (if linking isn’t the right approach.)