I tried to use Sakura LLM to translate some JP novels. It's really good and half the price of GPT3.5 turbo.
https://github.com/SakuraLLM/Sakura-13B-Galgame/tree/dev_ser...
410 karma · joined May 26, 2016
https://github.com/SakuraLLM/Sakura-13B-Galgame/tree/dev_ser...
However, the prompt is your business logic in most cases and put your business logic into a separate file make it harder to read and harder to maintain.
Chatbot should be part of your ui instead of replacing the ui.
https://github.com/flutter/flutter/blob/master/examples/plat...
I really hope flutter can provide a guide for FFI though.