What an incredibly interesting use of LLMs (generating text to practice typing). It leans in on what LLMs are good at. That said. I would love to see a middle tier pricing which had some features but avoided the AI use.
1. AI interactions cost the service money, which is inevitably passed on to the consumer. The if it's a feature I do not wish to use, I like to have options to avoid paying for that feature. So in this case, avoiding AI use is a purely economic decision.
2. I am concerned about the content LLMs are trained on. Every major AI has (in my opinion) stolen content as training material. I prefer not to support products which I believe are unethically built. In the future, if models can be trained solely on ethically sourced material where the authors have been properly compensated, I would think this position.
I've experimented in the past with running an LLM like this on a CPU-only VPS, and that actually just works.
If you host it on a server with a single GPU, you'll likely be able to easily fulfil all generation tasks for all customers. What many people don't know about inference is that it's _heavily_ memory bottlenecked, meaning that there is a lot of spare compute left over. What this means in practice is that even on a single GPU, you can serve many parallel chats at once. Think 10 "threads" of inference at 20 Tok/s.
Not only that, but there are also LLMs trained only on commons data.
Yeah, LLMs are indeed really good for this use case.
> That said. I would love to see a middle tier pricing which had some features but avoided the AI use.
Only paid features are AI features. Everything else is free and no ads :)
You can type anything and as much as you want, you have access to all the advanced stats, you can create a custom theme from a photo of your keyboard, etc.
Everything but AI features is free right now. (Might change in future as we’re adding a lot more features so we will definitely consider a mid tier price )