A ChatGPT-like assistant but private for developers | Hacker News Reader

A ChatGPT-like assistant but private for developers

autonomous.ai

42 pointstodsacerdoti31 comments

I tried hosting several small-size LLM models. It requires lots of work, but it’s worth it to customize my own sick chatbot. However, hosting a large model with billions of params is super painful and expensive.

Using a public service like chatGPT or Replicate is easier and more affordable but I am really concerned about data privacy. We all know some are collecting and using user data for training or other purposes. What makes you different from others?

Agree. That’s why I want to build a truly private AI, where users are confident to use without worry about their privacy. At first, we hosted stateless AI models using ephemeral processing to handle real time convos. We designed and implemented a protocol to ensure E2EE via TLS/SSL, GDPR/CCPA compliant with conversations stored locally on-device. We also provide an anonymous mechanism, no login required. Currently, we hosted open source flagship models of meta, Llama 3.1 405B, Llama 3.2 Vision and FLUX.1[dev] The product has just launched and is free to use with unlimited prompts. Paid plans are for AI agents. We still have a long way to go and would love to hear your feedback.

This is a really interesting approach to AI privacy. I'm curious to see how the performance of the local models compares to cloud-based solutions, especially when it comes to handling complex queries. Quick user feedback: I tried to generate violent images (just for the sake of testing), but I felt it was still being censored by the AI model.

We are utilizing the open source Llama3.1 405B model from Meta, which has shown impressive benchmark results compared to cloud-based alternatives. You are right, some AI models have their own built-in safeguard layers. You can try some Anon agent without built-in safeguard.

With Anon being marketed as a private, no-login AI assistant, I'm curious about the trade-offs involved. Does this privacy-first approach impact the assistant's capabilities compared to other more data-driven AI tools? Has anyone tested it out in real-world scenarios?

We have built this one with a privacy-driven approach and ofc we still have paid plans for that, which ensures that we don't collect user data with the "free options". You can explore the AI agent and pay to use your favourite one for a customizable experience

Privacy is the top one concern for Series B+ companies from my experience. On the other hand, if the LLMs need to run on a developer laptop you should choose small open source models which usually don't perform well. My experience with llama 8b was negative.

For individuals, we have an online Anon solution with privacy by design to help keep your data safe up to some extent. Series B+ companies might want to try our on-premise solution that keeps data locked down tight within their own space. No worries about your data leaving your control.

This approach to AI privacy is intriguing! I’d love to see how these local models perform on complex tasks compared to cloud-based alternatives, especially when it comes to efficiency and resource needs

Exciting to see a private AI tool with no logins and tracking. All the best!

That's a cool idea with private AI, it's rather interesting if it can store some metadata about previous searches for the next results being more accurate.

we use end to end encryption via TLS/SSL, comply with GDPR and CCPA and strict access controls to prevent unauthorized access. Conversations stay locally on the device, so you can defo view your previous data and searches

Can users delete their conversation history, and if so, how is it handled? Are conversations deleted permanently, or are they retained for a certain period of time?

Yeah you can delete your conversation history whenever you want. Since we store data locally only, you have full control over it. Delete it, and it's gone for good. We don't retain any copies on our servers.

That's really cool! How do you ensure the AI model is not storing or retaining user data?

This AI model is stateless, so it doesn't store any info about previous conversations. It processes input in real-time, in memory only and purges any temporary data as soon as it's done.

This is interesting, with e2e private chat, there are much more possibilities being unlocked without worrying about censorship

Yes we're using E2EE via TLS/SSL, storing everything locally, and keeping it all anonymous with no logins

Looks like an awesome tool! Privacy is always a huge concern, so it is great to see any AI company taking it seriously.

it comes from my concerns that today's chatbots gather user data and use them for training, even sell them without user awareness. That's why I want to build a AI, where users can feel confident using it and about their privacy

Need to understand, despite the privacy, is there any other difference compared to other AI assistants?

Besides our focus on privacy, what sets us apart is we'ew using some of the latest AI models out there. Our tech stack includes

- Llama 3.1 405B by Meta (outperforming GPT4 by 13%) - Llama 3.2 Vision by Meta (we're one of the first to use it!) - FLUX.1 [dev] by Black Forest Labs

Love the idea, going to try it out

Why does a furniture company do AI?

We're a team of engineers with a proven track record, having developed our first AI-powered product - a smart desk designed to boost productivity back in 2015 (https://www.youtube.com/watch?v=P56pkB-iEI4). Our vision is to create a portfolio of AI products, with AnonAI being one of those products.

Cool!

Thanks. Glad you found this one helpful

this is really cool

thanks! Hope you have a great time with Anon. Come back here some time and let me know how your chat goes

self hosting?

Traditional models: collect data and store it on central servers. Anon AI models: Conversational data still go to an LLM, currently hosted by us. However, we've implemented several measures to ensure user data security: - Our AI model is stateless, meaning it doesn't store any info about previous conversations. - We use ephemeral processing, where user input is processed in real-time and only retained for the duration of the request. - The AI model processes user input in memory only, without writing any data to disk. - Any temporarily stored data is automatically purged at the end of the process. As we scale with model providers, we'll thoroughly vet them to ensure they meet our security standards. Thanks for pointing this out, and I hope this helps address your concerns!

Reply on news.ycombinator.com