Hacker News Reader

Comment by ratg13 | Hacker News Reader

>a personal AI assistant to act as a second brain, and help me prioritize and remember things with human intuition

Everyone wants this, but this is not the product.

The current AI offerings are information in --> information out.

It is not meant to keep state long term, and it is not meant to be your friend. It is meant to answer questions with the information available to it.

You can even see in the example screenshot they showcase the fact that it is not designed to be asked follow-up questions.

View on HN

That's just the case for this bot, but is obviously not the case for ChatGPT.

I have a long running conversation with ChatGPT that I use to keep track of a verbal to-do list. I tell it my items with categories (e.g. work, personal, etc.) and estimated times, and then it outputs my complete task list, grouped by category. I then just tell it when I add tasks or complete tasks, and it continually keeps track of and outputs my current outstanding task list.

I've been using this for weeks now, and since it's all in a single conversation ChatGPT can keep track of the entire state over time.

I don't have access to plugins yet but it would be trivial to implement a personal AI assistant with ChatGPT if it could, for example, look up flight times and prices.

Oh dang, that sounds awesome. So a ChatGPT conversation doesn't have a historical limit? I guess I assumed it would start having to forget things at a certain point.

As far as I’m aware it will definitely forget things as the history grows. You can “remind” it about things to keep them relevant, and I think a cool product built like OP’s project would take that into consideration for long running tasks.

Yes, there’s a limit. Right now it’s 4000 tokens (gpt4 has a 32k model but i don’t think it’s available yet, at least with chatgpt). Once you near the limit, chatgpt will start dropping previous messages to stay under the limit. I don’t know what their algorithm is for deciding what to drop. It could be as simple as dropping the oldest stuff. Or maybe taking a long message and replacing it with a summary. But at a certain point, the conversation becomes “lossy”.

If it periodically outputs the latest state of the todo list, it won't have the problem of forgetting things from a long time ago.

FWIW that's exactly what it does (updates the current state of the whole todo list each time). After two weeks of use I've never had it make a mistake in keeping track of my total list of tasks. By default (I mean I didn't ask it to do this) it outputs the total estimated time for each category of tasks, and that summation has been wrong (which wasn't that surprising to me, as "LLMs aren't great at math" is a known issue), but even then I just tell it "Can you double check the category totals?" and it fixes them.

They might make it summarize intermittently behind the scenes when the context grows too large. Which would be how it has a gist of the stuff that was mentioned way earlier even thought it doesn't know the specifics.

And what happens when your window runs out? It prunes the oldest convo which you will need to keep track. Likely it could be fine, but if there was a task from a long time ago, that could get wiped out

I'm assuming at this point you'd need a means of pruning expired information from your list, and then importing that shorter list into a new session.

I guess the question is, can we get ChatGPT to make that new list reliably?

You could store whatever personal data you want in embeddings and use the api to refer to those. Can't do it on vanilla ChatGPT with the website though, I don't think.

Am I missing something or is that the opposite of how embeddings work?

https://platform.openai.com/docs/guides/chat - this API endpoint for completions doesn't take embeddings, just messages.

Their API docs for embeddings also don't talk about using them to get outside of the context size limit; instead, the way I've used it and seen others is "create embeddings from documents to enable fast search for relevant documents to populate in context" which still requires a separate data store.

Every time I add or complete a task, ChatGPT outputs the latest complete copy of my task list (I only had to ask it to do this the first time, then it just did that automatically). So, in other words, it always updates the latest "live state" in its most recent message. If I hit the limit on context window it would just take me ~30 seconds to open up a new window with my original prompt and the latest iteration of my task list if I so desired.

Here's where I posted a snippet of this convo a couple weeks ago: https://news.ycombinator.com/item?id=35390644

This looks way more like "improved chat interface to a Reminders app compared to Siri" or such than "prioritize and remember things with human intuition".

ChatGPT can keep some state for you, but there is a limit to the amount of tokens you can keep going in an instance.

It’s enough to keep a todo list going, it’s not enough to make it your friend / coworker

If you built what you were describing right now, either the flight questions would push out your todo list, or you would need to build something to keep state yourself.

You can use Langchain / Plugins to get around that issue

Reply on news.ycombinator.com