Hacker News Reader

Comment by a1j9o94 | Hacker News Reader

This is interesting. Can you expand on how this gets around the context window problem? Are you thinking the agent does a one-off task rather than continuing back and forth with the user?

I do think this will be way less than having all of the functions listed to begin with though. I think the discoverability is a novel approach. Honestly, I'm surprised ChatGPT with plugins doesn't do something like this by default rather than making you pick which plugins you want at the beginning of the conversation.

View on HN

First, the discoverability reduces the required context window. We don't have to explain every app we have, it's enough to tell ChatGPT one sentence about them and it will go deeper if it thinks that would help it to perform the task.

Also, we have not implemented it, we can have one or multiple level of managers just like at a company and each would delegate a task to a worker (who could also be a manager) and they would report back the result. Just like in real life, a manager doesn't have to know how something is done, it should only know it's done and the get the results.

We work for a large company and very likely have 100s of apps. We could build wrappers around them e.g. using selenium and we could interact with even old apps.

We could also do the same approach with databases. The db itself would have docs, each table and each field as well. So we could ask ChatGPT to query data from the db and it could fully understand the data before writing the sql query.

I've written about some hierarchical manager system with some friends when exploring how to use AI for larger set tasks. While the easy answer is simply using something with much larger context - `Claude` is amazing with an API key if you're on the waitlist - we definitely followed the same idea of splitting up the context into individual groups.

We had some success actually with layering another AI into the mix - having one AI look at a summary version of the context as a whole, and decide which pieces of context to assign to each manager. This of course requires a sidestep into another database of some kind to store the "master context" (AKA the full conversation, so you likely already have it in some form of storage), and of course a lot more calls to the AI which overall increases latency quite a bit.

1. Use an AI to provide a short summary of each piece of logical context and map it by access ID 2. Use another AI to determine which pieces contain the most useful additional context to the piece of the task being evaluated 3. Build the context from the generated ID list and pass to individual task manager AI

Reply on news.ycombinator.com