Show HN: I built an on-device LLM based chatbot for iPhones

I’ve been playing around with local LLMs for the past couple of months and decided to build something that can run on an iPhone. It’s a universal app built with SwiftUI and the excellent ggml library. The model is an SFT fine tuned and 4 bit quantised version of the RedPajama-INCITE-Chat-3B-v1 OSS LLM. It works reasonably well on recent-ish (~3 year old) iPhones, iPads and Macs. It was launched on the App Store yesterday[1] and Product Hunt today[2]. It seems to be reasonably ok at natural language interactions, but given its size, does pretty badly at coding and reasoning. Also, it sometimes hallucinates after a few back and forth messages. The fine tuning for natural language Q&A seems to have made the base model’s already bad performance on coding tasks, even worse. I’ll add a few App Store promo codes in a comment below, if anyone would like to try it out.

[1]: https://apple.co/43n6BsW [2]: https://www.producthunt.com/posts/personal-gpt