Hacker News Reader

Comment by zoogeny | Hacker News Reader

Once we start talking about multi-agent AI systems I think my intuitions go out the window. If we consider that GPT-4 is about as capable as a reasonably intelligent high school graduate or college freshmen, I wonder how much actual benefit we get from multi-agent setups.

I mean that intuitively I couldn't imagine replacing 1 experienced professional with 1, 2, 10, 100 or even 1000 intelligent high school graduates. Intelligence doesn't seem strictly additive across multiple individuals for all cases. But then I consider that one of the most powerful life-lines in the TV game show "Who wants to be a millionaire" was the "Ask the audience". I am reminded of the cliche of the wisdom of crowds, even when the crowd is made up of non-experts.

This suggests to me that there is a kind of problem where multiple lower power agents can solve the issue to a higher quality. But there are also kinds of problem where a single higher-power intelligence will be necessary. I haven't developed an intuition when each approach is valid.

View on HN

I suspect the real value is more from an architecture and caching perspective.

Architecture as in you may have multiple GPT instances being prompted to look at a problem from different angles: "Analyse the problem as a pessimist", "As an optimist", "as a mathematician", "an engineer", "a philosopher", etc.

Caching, as in determining what you can store from these outputs, e.g. "give a numeric 1-10 score of what you think of this product as a LGBT-friendly conservative from the Midwest", etc.

> look at a problem from different angles

Sure, but just as a thought experiment imagine you have a really bad disease. I give you two options: 100 high-school students can diagnose and prescribe a treatment or you can choose 1 professional with 10 years of experience in related diseases.

You might initially prefer the professional with 10 years of experience. Would your opinion change if I told you that I selected the high-schoolers to be diverse so that one is pessimist, one is an optimist, one got good grades in engineering, one loves philosophy?

Of course, my intuition might be wrong. For example, perhaps it legitimately would be better to have 100 high-school students where one is a high-school level ability with ear-nose-throat, one is a high-school level ability in oncology, one is a high-school level ability in cardiology, etc. Except some control AI would have to synthesize their answers into a coherent response ... and that controller would be high school level.

I'm not really sold either way if I am honest. I don't think we have the answers to these questions. It just shows that my intuition about intelligence is open to challenges.

In real life, though, that specialist with 10 years experience has a 2 year waiting list for new patients, and you wouldn't even get in to see them anyway, because they don't take your health insurance.

Meanwhile, the "100 high school students" on a mobile phone are the only medical consultation someone in Sub-Saharan Africa or Southern Asia is going to have access to at all.

"Why Not Both" would surely apply here. Maybe we can't give everyone access to a kind, caring, patient human doctor; but we sure can make coming in second place a lot less painful.

[0]https://www.who.int/news/item/13-12-2017-world-bank-and-who-...

I suspect the divide may end up similar to that between a start up and a large company. Yes, some problems are better solved by a single person. But others benefit from a division of thought even if there is some bureaucratic overhead.

That is my speculation. But the question is one of intuition. If the speculation is correct, that some problems are better solved by a single more intelligent agent, then how can I determine the appropriate approach?

What I mean to say is, if I am considering building a product based on LLMs then I may have to make a basic decision: can I use multiple cheap LLMs in a multi-agent setup or must I use a single expensive powerful LLM. Right now I don't have any intuition on what kinds of problems are solved most efficiently by either approach. Just looking at a problem description I can't intuit which approach is appropriate.

This is a very interesting and important question. I think that until we answer your question rigorously, the best bet right now is to start with the expensive powerful LLM, and then after your system is prototyped and working, piecewise move prompts in your prompt network over to other systems as it proves feasible.

If you try to start with the multiple cheap LLMs approach, it is almost certainly going to be more difficult to get the prompting right, and if you don't already know what you are trying to accomplish is feasible, you're adding a lot of work that might add up to nothing (even if it would actually work if you used the more powerful model).

It's not quite the same thing, but the "mixture of experts" model is a popular deep learning approach that displays very good performance: see e.g., https://arxiv.org/abs/2208.02813

Do they have access to all of the information on the internet? Might give them a lot better chance.

The difference is that an LLM isn't 1000 different intelligences. It's one intelligence, being asked to pretend to be 1,000 different people. Every instance is the essentially the same weights trained on essentially the same data. The difference in perspective doesn't resemble that of the difference between any two humans.

Humans love to think of multi agent systems as being like a team of people. It's much more like a writer imagining different characters and how they would respond. When George RR Martin imagines all 500 characters in Game of Thrones, there is a lot of diversity of perspective and thought there. But all of that is coming from one intelligence and doesn't represent a collaboration in any traditional sense.

>The difference is that an LLM isn't 1000 different intelligences. It's one intelligence, being asked to pretend to be 1,000 different people.

No it's not. It does no good for a Language Model to configure a global persona. It needs to be able to predict text from wildly varying backgrounds and contexts. It's not pretending anymore than anything else it does is pretending.

That's why experiments like the below actually work

Large Language Models as Simulated Economic Agents: What Can We Learn from Homo Silicus? (https://arxiv.org/abs/2301.07543)

Out of One, Many: Using Language Models to Simulate Human Samples (https://arxiv.org/abs/2209.06899)

A perfect LLM would predict Einstein as well as it would predict the dumbass down the street.

Now RLHF does incentivize a more global persona by default but stepping away from that is trivial

I imagine you could still build a dream team of GPT3.5, GPT4, LLama2, Vicuna and their friends

This is something I’m working on, for now in a much more simplified format and still very rough. But I did manage to get team members to be contextually aware, so they know they team and everyone’s details. They can even search through chat threads with any given team member. https://www.conjure.team/

> The difference is that an LLM isn't 1000 different intelligences. It's one intelligence, being asked to pretend to be 1,000 different people.

No, its not.

Its more like 1,000 individuals sharing one genetic template.

Its just that the experiential context that along with the template makes an individual is very small with LLM instances.

Cosign, with extreme prejudice. We went through this with AutoGPT.

I see two camps, people quietly working, and people working on grandiose ideas from the initial rush that make good headlines but not good products.

Some informal warning signs I use subconciously:

- "paper" on Arxiv, and its about prompt engineering

- agent_S_

- state machine where the LLM is eating its own output repeatedly

- LLM makes decisions based on its own output.

There's 100x more alpha in the obvious stuff because even that's not well-implemented or shared widely. Ex. people are still stuck on hallucinations 90% of the time: an obvious way to handle that is doc retrieval.

Last n.b.: in 2021 I was frustrated with ~100% of models being behind locked doors. Then, I saw how GPT3.0 was working and evolving. My mantra became "products not papers." Maybe that applies downstream of the LLM now.

> My mantra became "products not papers." Maybe that applies downstream of the LLM now.

I agree, in the sense that my intuition is fed by having as many available examples as possible. I am not even sure one could quantify the difference between intelligence levels in a way that I would be satisfied with in a paper.

For example, I find it intuitive that a mixture of 10 narrowly fine-tuned GPT-3s are better at a task than 1 broadly fine-tuned GPT-3. But I don't have a real intuition about how many GPT-3s you would have to mix to match the quality of GPT-4, or if there even is any number of GPT-3s you can mix to achieve the result of GPT-4. I think we just need to start building systems and see what happens.

Are you saying AutoGPT has similar problems or has it solved it?

We tried it recently and found it very challenging to get it to accomplish simple tasks.

TL;DR: AutoGPT has similar if not identical problems

re: AutoGPT

Got really excited at first. Thought maybe I had missed that it was viable in a year of playing with LLMs. Tried a few demos, didn't work. Looked into it more and confirmed a core loop involved LLM eating its own output to make decision.

I still kept investigating it on my todo list, in case my earlier experiments with that approach were wrong.

I took it off the todo list later.

I saw near-universal feedback like yours, that general technique never worked IMHO, and IMHO sycophancy explains why. Paper here[1], TL;DR the model is very likely to agree, so critical feedback loops over multiple steps tend to settle into a loops of steps.

IMHO this doesn't mean sycophancy breaks _all_ workflows, ex. a flow for writing a story involving outlining, writing, criticizing, then rewriting is a genuine real quality boost.

However, if a human does write => criticize 10 times, it keeps getting better each iteration. If you have an LLM do it 10 times, IMHO it's actively harmful after round 3.

"outline" => ["write page 1", "write page 2", "write page 3"] => ["feedback on page 1"..."feedback on page3" => ["use feedback and original draft to rewrite page1"..."page3"] => "combine pages 1 2 and 3 into cohesive story"

[1] https://www.anthropic.com/index/discovering-language-model-b...

I've tested the writing loop you describe and the biggest problem seems to be that currently GPT4 is good at generalities, but gets into loops of generalities that doesn't drive things forward when you expect too much detail.

It's not that it can't improve the writing further per se, but that it takes very detailed prompting to get it to give a detailed enough critique to do so consistently enough across even a page (e.g. ot might come uo qith a great lone but proceed to edit out the best paragraph elsewhere) to the point that I tend to agree with you in as much as it at least will take a much more convoluted chain of prompts to maybe get there at the moment, and you'll be fighting GPT4s tendency to actively cheer on really juvenile prose the whole way.

In a way I think the biggest hindrance to get it to write better at the moment is that it has awful "taste", and having to explicitly give it a long list of rules to check against is a poor substitute.

As an aide, though, I think you could get reasonable but not great results at "bridge these two paragraphs and maintain the style" type tasks, or expanding descriptions into a paragraph or two, though more so for non-fiction writing.

For fleshing out the basics of a technical spec and pointing out what I've missed I've had decent luck, on the other hand. It's not come up with any Earth shattering revelations, but for a dry spec that's not the point.

> However, if a human does write => criticize 10 times, it keeps getting better each iteration. If you have an LLM do it 10 times, IMHO it's actively harmful after round 3.

Telephone (game) https://en.wikipedia.org/wiki/Telephone_(game)#Game :

>> The game has no winner: the entertainment comes from comparing the original and final messages. Intermediate messages may also be compared; some messages will become unrecognizable after only a few steps.

Transmission chain method: https://en.wikipedia.org/wiki/Transmission_chain_method :

>> The transmission chain method is a method used in cultural evolution research to uncover biases in cultural transmission.[1] This method was first developed by Frederic Bartlett in 1932.[2][1]

Feedback > Electrical Engineering; Positive Feedback, Negative Feedback,: https://en.wikipedia.org/wiki/Feedback#Electronic_engineerin...

Control Theory > Stability: https://en.wikipedia.org/wiki/Control_theory#Stability

Multi-agent System > Concept: https://en.wikipedia.org/wiki/Multi-agent_system#Concept

Convergence (disambiguation) https://en.wikipedia.org/wiki/Convergence

Convergence (logic) https://en.wikipedia.org/wiki/Convergence_(logic) :

> In mathematics, computer science and logic, convergence is the idea that different sequences of transformations come to a conclusion in a finite amount of time (the transformations are terminating), and that the conclusion reached is independent of the path taken to get to it (they are confluent).

> More formally, a preordered set of term rewriting transformations are said to be convergent if they are confluent and terminating.[1]

And then

Consensus (disambiguation) https://en.wikipedia.org/wiki/Consensus_(disambiguation)

Revision is a step in a Writing Process.

Revision (writing) https://en.wikipedia.org/wiki/Revision_(writing)

Writing Process: https://en.wikipedia.org/wiki/Writing_process

Collaborative writing > Types, Tools: https://en.wikipedia.org/wiki/Collaborative_writing#Tools

And now it is time to underline the premise(s) and the conclusion(s), time to apply Critical Thinking; Logic and Rationality.

Critical Thinking > Logic and rationality: https://en.wikipedia.org/wiki/Critical_thinking#Logic_and_ra...

To simulate a full-scale multi-agent game, there would need to be a "Fourth Estate" (and maybe a Fifth Estate); a peanut gallery of dissenters with signs and no jobs.

Then, you could model consensus with LLMs and have something better than the mediocrity that sometimes results from committees.

More samples from the same LLM vs Sample different LLMs

Maybe feed one or more agents relevant encyclopedia articles as context first; which read the most encyclopedia articles first?

Mmmm… Assuming you’re referring to US high school, I’m going to say that GPT-4 is more useful than a high school graduate.

Rationale:

- Speed - Ability to recall - Depth and breadth of “exposure”

I can’t speak to others high school experience, but in my experience, high school does not prepare young adults to be useful in the same settings that GPT-4 could be useful.

Example prompt: - Pose this to a high school graduate and GPT-4, somehow benchmark quality.

1. “You are an experienced business strategist. Your client has the following problem <insert problem synopsis here>. Generate an outline of brainstorming topics your client should consider to accomplish <insert goal here>.

- First, GPT-4 is going to respond in ~30sec or less - Second, a high school grad has 0 context for these types of prompts.

> Mmmm… Assuming you’re referring to US high school, I’m going to say that GPT-4 is more useful than a high school graduate.

Useful for what?

LLMs are largely useless for the vast majority of labor that people with HS degrees tend to do (which mostly involves not being behind a keyboard all day).

Forgive my conjecture, as I'm no expert in this field, but have enjoyed playing with the different models.

I see it as a matter of attention. GPT-4 is limited in the number of tokens you can feed it and receive back from it. I see this as how much attention the LLM can give you. I see meta-gpt and other models like this as increasing the attention of the LLM by allowing it combine it's short attention span with multiple assessments of the request, and give you a more complete picture of what you want because it can keep giving it consistent attention to the problem at hand, instead of simply trying to solve it on the first try.

Someone please correct me if I'm way off, but this simple mental model helps me abstract this.

The problem with this is that it has no memory across the different contexts. An analogy would be giving one page of a five page document to five different people, then taking it away and asking them to collaborate. While they can each give more attention to their individual page, none of them can see the whole picture and a lot of information will be lost when trying to communicate.

You can use multiple agents, or split a lot of information across multiple requests to one agent. The result is the same. Some problems require a full understanding of the whole picture.

That’s a great analogy thank you!

> If we consider that GPT-4 is about as capable as a reasonably intelligent high school graduate or college freshmen, I wonder how much actual benefit we get from multi-agent setups.

It's worth noting that GPT-4 internally uses a Mixture of Experts (MoE) model with 8 'experts' internally, so it's more similar to a multi-agent setup than you might think initially.

> It's worth noting that GPT-4 internally uses a Mixture of Experts (MoE) model with 8 'experts' internally, so it's more similar to a multi-agent setup than you might think initially.

That's a very common misunderstanding of what MoEs are. What you're describing is an Ensemble.

A MoE is where a the input is "routed" to one of X "experts" in the higher layers of the network. Something like this: {input} -> [lower layers] -> <decide which upper layers to send input to> -> [expert] -> output

And these experts aren't experts in the space of human defined subjects. Their expertise are in some embedding space, which might or might not line up with our intuition of a subject.

Right, but the intuition I am going for isn't "Are 10 high-school students who collaborate together smarter than 1 high-school student working on their own".

The intuition I am asking for is more like: "Are 10, 100, or 1000 GPT-3s capable of producing the equivalent intelligence of 1 GPT-4". That kind of reasoning expanded to GPT-N.

And further, if no number of GPT-3s can reach the level of intelligence of a GPT-4, then what set of problems can be solved by 100s of GPT-3s in a mixed model and what problems require the single GPT-4.

You can generally amplify models by using more tokens, more calls or relying on some oracle of correctness. I see rapid growth of synthetic datasets from now on, we need the diversity of synthetic data to progress further.

> It's worth noting that GPT-4 internally uses a Mixture of Experts (MoE) model with 8 'experts' internally

Has this actually been confirmed, either officially by OpenAI or otherwise? As far as I know, George Hotz claimed this once, and since then everyone just assumed it was the truth without actually waiting for any sort of verification.

TL;DR This might work - but it will be like watching Groundhog Day. It will require many iterations, make too many mistakes to get there and won’t remember a thing.

My naive understanding of layers in a model is that each layer loosely acts as an expert in one step of the entire process.

For example, in an object recognition model, one layer takes on the task of separating objects from the background, another excels at knowing the colour of different things, another might learn the difference between a blue sky vs. the colour sky blue.

So essentially, we’re trying to mimic the same working model at a higher level of abstraction. Similar to how our body is made of atoms. Many atoms make a molecule. Many molecules make organic tissue, and amino acids that perform more complex operations. Skip all the way up and you have a human being. Human beings in numbers can put a man on the moon, create anti-matter and nuclear explosions.

In theory - this can work just as well. One agent to provide a solution, another to critique it, another to verify it, another to mimic the end-user. The big obvious missing piece is memory and the ability to learn while doing (at least in existing LLMs).

It’s like having a software development team that’s frozen in time and knowledge. These LLM agents will always require some micromanagement and hand holding, and will waste a lot of resources with failed attempts - that they are going to repeat every time you ask them to perform this task.

GPT-4 is not comparable to a high school graduate.

It will happily hallucinate facts and claim they are true. Well, ok, maybe it is ;)

The issue they seem to tackle is trying to minimize hallucinations by injecting some human expertise into the pipeline. They do this by more strictly defining the roles and tasks a step in that pipeline needs to accomplish.

> It will happily hallucinate facts and claim they are true. Well, ok, maybe it is ;)

As if humans don't, bullshitting:

https://www.sciencedirect.com/science/article/abs/pii/S00221...

https://www.poynter.org/fact-checking/2018/this-study-is-all...

Interesting, multi-agent was almost my first thought when I began using ChatGPT. For complex tasks I figured you would want to chain together several LLMs, each for a dedicated domain. It would actually be really neat to be able to use a Zapier-like interface to build your own AI workflow.

Understanding a single node of the system is not the same as understanding the sum of all parts. In other words, a system of 2 high school graduates may appear at first as the sum of the output of just 2 high school graduates but the sum of 1000 high school graduates may appear as super intelligence.

The phenomenon I am describing is known as "Emergence."

> I mean that intuitively I couldn't imagine replacing 1 experienced professional with 1, 2, 10, 100 or even 1000 intelligent high school graduates

This analogy doesn't make sense, because the professional is presumably also a high school graduate. This case is more like leveraging a team of specialists with expertise in different domains.

I think it has to do with the fact that LLM's are stochastic, autoregressive models. There are "trajectories" that it can take that are wrong (hallucinations) and there are trajectories that lead to the right answer. In essence multi-agent is doing something like self consistency, self-reflection.

It’s the power to persuade (others to follow you). How do you decide which of the 100 agents is the expert ? They need to persuade the others. especially when they don’t know if they are the expert themselves. sometimes the right decision is to follow someone else.

Different components of thought should be specialized. Choosing the meta structure will likely have an impact on what the system can achieve simply by the benefit of compartmentalization.

Reply on news.ycombinator.com