Looking to build a personalized AI for myself. Any resources? | Hacker News Reader

Looking to build a personalized AI for myself. Any resources?

23 pointseerop11 comments

Hey, I'm looking to create a personalized AI for myself in the context of me in an online community. I spend a lot of time in an online community, and I'd like to have a personalized AI to help me with communicating there. I want to have the AI know all the members there and give me ideas for conversation topics.

I know I could probably do this with ChatGPT, but I'm wondering if there's a specific solution built for this.

Thanks!

View on HN

What you want to look into is retrieval augmented generation (RAG). Basically you store a bunch of facts as text in a vector database, then you query the vector DB for things to stuff into the prompt that you send to the LLM.

What you write to the LLM might be "give me a conversation topic I might have with a member of the community", and your vector DB will have a bunch of things like "Dumbledore is a member of the community. Dumbledore likes to talk about magic". In theory the vector DB search will turn that up, and then it gets stuffed into the prompt so what actually goes to the LLM is something more like "You are a helpful AI assistant. Dumbledore is a member of the community. Dumbledore likes to talk about magic. give me a conversation topic I might have with a member of the community".

The heavy lifting will then be collecting and storing all of the data that goes into the vector DB.

Thanks a lot. Do you happen to know of any products that make it easy to implement this without being too technical?

The various langchain libraries out there make it easy for developers, and they include examples.

Before getting deep in Rabbit holes and paying $1k to fine tune something, consider using OpenAI assistants API. You can experiment with it in the Playground without writing code and that itself might be enough! Or write some code to put your UI over it. Then once you have digested your own dog food you might have a product to sell too!

Just to correct the record a bit, while you certainly can spend $1K+ to fine-tune a model we've found that the average user on openpipe.ai spends less than $30 per fine-tune. It actually takes quite a lot of data (likely more than the OP has produced in their interactions with the forum) to crack $100 in fine-tuning costs!

Of course this is heavily dependent on what you're using for source data. And in any case, the OP would certainly be spending far more than $30 in time trying to get their data in the right format, which may or may not be a good use of time based on how excited they are about learning the process end-to-end. :)

Would a model actually perform better fine tuned on such a small data set? Seems to me that just giving it the data during inference of new stuff would net better results, using something like RAG.

I see here a lot of adverse answers but what OP is asking for will be common for all of us in the short term. It is just that now relatively few players "game" the system using bots. Even if we don't like it it is happening and cannot be stopped.

If I am a sales guy I would like to have my communication processes by an assistant, human or not. It is not about spam but the difficulty of handling simultaneous conversations with even legitimate contacts.

You want help building a social media bot?

I bet this guy also has an idea for an app/game and he just needs you to help code it

So you want to have an AI help you participate in a community so you can participate in the community less?

Dont be a spammer. Problem solved.

Reply on news.ycombinator.com