CustomGPT | Hacker News Reader

CustomGPT

customgpt.ai

37 pointsowen_fay31 comments

Does anyone have any pointers as to how I can train ChatGPT on my personal corpus and create my own personal assistant?

Just shove all your information in the prompt. I assume that's what these guys are doing anyway. ChatGPT doesn't yet have any API, but you can use text-davinci-003, which does have an API and is functionally identical.

The API costs for a large prompt are quite high, and since you have to repeat it with every single query, your costs can go up very quickly.

Gotcha! Thanks for the response. So if I have, say, 10 PDF files which I plan to extract text from and use to fine-tune davinci, I would have to supply the entire textual information extracted from the PDF file each time?

davinci-003 has a 4k tokens cap, you cannot just 'shove all your information in the prompt.' Can parallelize but then rate limits become a problem. Really curious to hear how these guys are handling it.

You shove the whole corpus in a vector db using embeddings, query the nearest neighbors to an input and inject those into a prompt to pass to GPT.

https://langchain.readthedocs.io/en/latest/modules/chains/co...

I thought fine-tuning allows you do "inject the prompt" only once. https://platform.openai.com/docs/guides/fine-tuning

Ah, the rub is that a fine-tuned Davinci costs 6x a standard Davinci. Interesting. https://openai.com/api/pricing/

Check OpeanAI's tutorials [1] and cookbook [2]. Or try LangChain [3], which abstracts a lot of the work.

[1] - https://platform.openai.com/docs/tutorials [2] - https://github.com/openai/openai-cookbook [3] - https://github.com/hwchase17/langchain

This is great. Thanks!

It's insanely easy to use RelevanceAI to upload a 1-column spreadsheet, click the "vectorize" button, and then have something working. The problem is putting the data into that spreadsheet.

https://relevanceai.com/blog/use-all-of-your-blog-posts-and-...

As far as the next step...

Google allows you to export your email data. Ideally, you could download that data and upload it to a service that looked at original messages and your replies (and replies to replies) and trained on that file without having to make sense of what Google gives you.

If that was working, then you could have it reply to emails for you.

Looks similar to what we built! https://www.getbuff.io/

We're taking a more open source approach to get feedback from others in this space since it's evolving so quickly.

We've actually been using Buff as a chat bot on our Discord server to iteratively explore vulnerability data (CVEs) using AI.

For example:

> @Buff What is Log4Shell?

>> Log4Shell is a vulnerability named by LunaSec. It affects users of the log4j packages, primarily in Java but also in Scala, Groovy, and Clojure. "Log4Shell Guide: How To Detect and Mitigate the Log4Shell Vulnerability (CVE-2021-44228 & CVE-2021-45046)" is available on LunaTrace.

>> SOURCES: https://www.lunasec.io/docs/blog/log4j-zero-day-mitigation-g...

The idea is that we can ingest unstructured data from GitHub Security Advisories (and the links they link to, like that LunaSec blog post) and then "query" that dataset with natural language.

If you'd like to play around with it yourself, feel free to join and try it out in our Discord. https://discord.gg/dtGHzZK6Cq

(It only has a few vulns of data in it right now and we're scraping the data to train it right now. Any ideas that others have to share would be awesome!)

Founders really need to start doing 10 minutes of due diligence to ensure that their sites won't crash and burn under moderate Hacker News or Product Hunt traffic before posting at these places. You get one shot at front page success, and are now losing valuable impressions because people can't even open your website. And you not being able to keep a marketing page up and running doesn't really give me confidence in adopting your actual product. A few clicks in Cloudflare could have easily prevented this.

> A few clicks in Cloudflare could have easily prevented this.

In my experience, this is surprisingly harder to achieve in practice than it sounds. I've had to spend quite a lot of time figuring out CloudFlare's arcane composition of page rules that would prevent a simple web site's denial of service by a simple netcat command.

That says more about Cloudflare than you think it's saying about you, tbh. I see this often, many people enlist their site into Cloudflare and consider the job done, and Cloudflare doesn't necessarily discourage that either until their app draws traffic and suddenly sales is your new neighbor.

There's a very slim chance of hitting the front page on HN (believe me, look at my post history, I tried a couple of times), do you prepare for such an unlikely event...

I mean if course you should, because otherwise why post on HN, but if it's 'just' a side project, I understand why you don't want to spend time configuring Cloudflare.

Looks like this has been hugged to death by HN. Does anybody have a mirror of the site they could share?

Am I too harsh for expecting that in 2023, a company actively signing up customers for a cloud-based service should have enough autoscaling so that getting to the front page of HN doesn't overload the landing page? It just doesn't give me much faith that the actual product will be stable enough to use. Especially just giving a generic 5XX error from nginx is a bad sign.

Edit: did this get manually knocked off the front page without getting flagged?

I feel like it's only the part with generic nginx 5xx error that sets the bar a bit too high. Too many huge services suffer from the same sin.

Let's be fair, ChatGPT has had issues with being overloaded ever since its initial release, and they're still not entirely fixed.

ChatGPT may need a tad stronger infrastructure than a landing page.

Sure, but their landing page never goes down when ChatGPT is over capacity, and their over capacity page is a delight of downtime design. It has auto-generated content in different genre styles about being over capacity. They even let you enter your email to get notified when it is back. But I do get frustrated that every time I've entered my email to be notified I've never actually been notified.

Or could this be a marketing stunt? OMG we are so popular that nobody can get to our site for the first 48 hours etc. etc.

Page is down, but here is a description of the service by the founder himself:

https://www.producthunt.com/posts/customgpt

Does anyone knowledgeable in AI know how feasible the claim of training your custom ChatGPT really is? Because this would be the holy grail we’ve all been waiting for, no?

I wonder if it's actually "trained" or is it rather "imagine you've generated all that fancy text content, which follows now: ${content}".

I also though of this (prompt engineering), but I think that would hit the token limit pretty fast.

My guess is that they are simply finetuning the model on your provided data: https://platform.openai.com/docs/guides/fine-tuning

Training a custom chatbot is easy. Having it reliably retrieve the data you want is hard. For any given question, you can get it to look at a maximum of 4k tokens (16k characters, or 3k or so words). So when you want your chatbot to have access to a LOT of data, you usually end up having to work around that - typically by:

1. chunking up your documents into ~2k word chunks, 2. and then getting the query, doing a search using that query, and getting the top N results back, 3. running the prompt on each result, and then 4. gradually combining the results of each run of the prompt.

Each step is a little lossy, and there are quite a few steps there, so the bigger the corpus gets, the lossier/less useful your bot is.

Now that being said, there are ways to improve that, but for now at least, its unlikely that a single slackbot is going to solve all your knowledge management problems for you.

I did a similar thing where i had a specific "journal" chat with CGPT and gave it a biography and primed it to act like Socrates with mental health training, and counselor skills. I get it to give prompts and also ask questions that promote elaboration.

Actually journaling for the first time in my life, it is amazing when it remembers things that i have told it and is super motivational, especially since it is always available and never gets bored of me asking for advice or venting about my day.

It's funny that their "brain" logo is literally two cloud icons rotated 90 deg and stacked together :)

CustomGPT lets you "Build Your Own ChatBOT" with your own data. Using OpenAI's language models, this chatbot lets you write blog posts, answer customer service questions, plus 100+ use cases.

Oh man I can't wait to argue with my zettelkasten.

Reply on news.ycombinator.com