Hacker News Reader

Comment by iamflimflam1 | Hacker News Reader

Was thinking of building something very similar - upload your technical docs - and create a chat bot help system.

One of the tricky things to overcome is the length limits of the prompt you can feed into things like GPT-3. There are some suggestions on the OpenAI website on how to overcome this. The main one seems to be to filter the content using embeddings and then only feed "relevant" sections into the prompt.

Would be interested to know what your approach is.

View on HN

I've been working on a hobby project to make all the site, videos and conversations I've seen searchable (aka external memory). While building it I used this to prepare the data and get around the prompt limits https://github.com/jerryjliu/gpt_index

> https://github.com/jerryjliu/gpt_index

Could you share anything (e.g. how many rows of data and tokens in each row) around how much it cost you to use GPT Index? It looks interesting, but it seems it'd be expensive.

Looks like it’s making extensive use of embeddings. The embeddings and vector comparisons are dirt cheap. The summarizing / answering functionality when the LLMs get involved is going to be the expensive part.

Is there a list of GPT+search projects? I've been seeing them popping up more and more.

Someone can create a chat bot which is fed HN comments and it tells you which GPT+search bot to use for your usecase

this is where the future is

Wouldn't your energy be better spent doing a better hierarchisation of your doc so users can navigate to the authoritative anwser easily and intuitively? Rather than relying on some black box commands to get the information that should be what people look for but maybe not?

I have been responsible for maintaining many thousands of pages of documentation with a dozen+ staff, and can say that at least for my industry there is often no hierarchy that fits every use case. Then consider that your documentation references dozens or hundreds of other pieces of external industry documentation, and now you have a real mess. That’s before we even get into the ERP system and all the relevant material specifications and such that’s stored away in there.

I do think LLMs will eventually be the solution to this long-standing problem. Even if you could ask questions and it would do a general summary for it and then point you in the right directions for more details we’d be way ahead.

I was able to break down the content into smaller prompts/completions. And remove some recurring and low-information data. Your suggestion of using embeddings to identify and feed relevant completions is pretty good. I'll try that soon.

Here’s the example notebook that shows how to do this.

https://github.com/openai/openai-cookbook/blob/main/examples...

What's the main difference between using https://trainmy.ai/ vs. with your app.

Since I'm planning to use that for where i work, for the same use-case.

Hi, I took a quick look at trainmyai. I would say we are focused on this specific usecase. Making it easy for users to interact with help and support docs and knowledge. We will keep improving and getting better in this specific use case.

You get custom/fine-tuned models with Twig that only you have access to. You also get to train on private/behind-login information that is available to only your users.

Think of it as use-case-specific solutions vs generic solution.

From what I was thinking of bulding:

Packaging it up as a nice widget that can be embedded on your site easily.

So you just upload your docs and have a snippet of JavaScript for your website that gives you a help bot.

Agree, a snippet that you add to the application that trains every day and gets better. I am thinking we can ingest data every day/week as documentation updates.

Would be good to have a chat - sounds like we are thinking along similar lines. My email address is in my profile.

Reply on news.ycombinator.com