Hacker News Reader

I tried hosting several small-size LLM models. It requires lots of work, but it’s worth it to customize my own sick chatbot. However, hosting a large model with billions of params is super painful and expensive.

Using a public service like chatGPT or Replicate is easier and more affordable but I am really concerned about data privacy. We all know some are collecting and using user data for training or other purposes. What makes you different from others?

View on HN

Agree. That’s why I want to build a truly private AI, where users are confident to use without worry about their privacy. At first, we hosted stateless AI models using ephemeral processing to handle real time convos. We designed and implemented a protocol to ensure E2EE via TLS/SSL, GDPR/CCPA compliant with conversations stored locally on-device. We also provide an anonymous mechanism, no login required. Currently, we hosted open source flagship models of meta, Llama 3.1 405B, Llama 3.2 Vision and FLUX.1[dev] The product has just launched and is free to use with unlimited prompts. Paid plans are for AI agents. We still have a long way to go and would love to hear your feedback.

Reply on news.ycombinator.com