Hacker News Reader

I wanted a privacy preserving AI chat bot since I didn't feel comfortable about putting personal issues into ChatGPT. It uses webllm models to run in the browser. Make your own chat by saving a message.

Disclaimer: I'm the creator of https://hony.ai that was used to build this demo.

View on HN

Ok, but how is Marcus Aurelius involved? Does it use search the Meditations with RAG, or does it use a model fine-tuned on extracts from the Meditations? Or...?

You can make any chat you want by saving the last message, e.g. if you just need a friendly chat bot, you can create that as well.

Ah, a most excellent inquiry, my dear! adjusts philosopher's cloak As Marcus Aurelius, I must say that the connection between Marcus Aurelius and this chatbot is purely nominal.

Cool! Can webLLM handle inference of models with any meaningful size?

Can I ask what model is used?

Thanks! It's using Llama 2 7B, It supports bigger models but those take longer to download and also infer (if run at all depending on the device)

Reply on news.ycombinator.com