Where to leave feedback? I am trying the Mistral dolphin model but getting GGML ASSERT errors referencing Users/tito lol (not me). Using iPhone 14 Pro Max.
I’ve just submitted a new update for review with a few small but hopefully noticeable changes, thanks in no small part to your feedback:
1. StableLM Zephyr 3b Q4_K_M is now the built-in model, replacing the Q6_K variant.
2. More aggressive RAM headroom calculation, with forced fallback to CPU rather than failing to load as you observed, or crashing outright in some nasty edge cases.
3. New status indicator for Metal when model is loaded (filled bolt for enabled, vs slashed bolt for disabled.)
4. Metal will now also be enabled for devices with 4GB RAM or less, but only when the selected model can comfortably fit in RAM. Previously, only devices with at least 6GB had Metal enabled.
Thank you so much for taking the time to test and share your experience! Feel free to reach out anytime at britt [at] bl3 [dot] dev.
Britt