Generative Agents: Interactive Simulacra of Human Behavior
arxiv.org
arxiv.org
To directly command one of the agents, the user takes on the persona of the agent’s “inner voice”—this makes the agent more likely to treat the statement as a directive. For instance, when told “You are going to run against Sam in the upcoming election” by a user as John’s inner voice, John decides to run in the election and shares his candidacy with his wife and son.
The scene: https://youtu.be/ZnxJRYit44k
Found it! : https://medium.com/swlh/bicameral-mind-humanoid-robot-with-g...
Imagine how well they will manage up, given human managerial behavior just becomes a useful prompt for them.
Fortunately, they can't be told to vote. Unless you are in the US, in which case they can be incorporated, earn money, and told where to donate it, which is how elections are done now.
Seriously. Scary.
On the other hand, if Comcast can finally provide sensible customer support it's clear this is will be an historically significant win for humanity! Your own "Comcast" handler, who remembers everything about you that you tried to scrub from the internet. Singularity, indeed.
I already pointed out they can influence elections with money.
And bots are already used to influence on social media. AI bots are going to be insidious.
I think that’s pretty extreme hyperbole.
A temporary exception would be the economically still incentivized disruption of the environment. I say temporary, because at some point it will stop, by necessity. Hopefully before.
But I can relate to the deep frustration you are expressing.
--
The problem isn't individuals, for the most part. The problem is that we build up systems, to provide stability and peace, and to be more just and equitable, by decentralizing the power in them. That way the powerful can't change them on a whim. (Even though they can still game them.)
But this also makes them very resistant to change.
Another effect is that as systems stabilize myriads of seemingly unimportant aspects within themselves, that stability represents the selection of standards and behaviors that give the system its own "will" to survive. That "will to survive" is distributed across the contexts and needs of all participants.
So any pressures to make changes, no matter how well thought out, encounter vast quantities of highly evolved hidden resistance, from invisible or unexpected places.
Even the most vociferous critics of the system are likely to be contributing to its rigidity, and proposing incomplete or doomed to fail solutions, because all these dynamics are difficult to recognize, much less understand or resolve.
--
My view, is that this cost of changing systems needs to be accepted and used to help make the changes. I.e. get all the CFO's of all the major fossil fuel companies in a room. Establish what kind of tax incentives would allow them to rationally support smoothly transitioning all their corporate resources from dirty energy to clean energy.
It would be very expensive. It would look like a handout. Worse, even a reward for being a bottleneck to change.
But they are the bottleneck precisely because of all the good they have done - that dirty energy lifted the world economy. And whatever it cost to "pay them off" would be much less than not paying them off.
--
The costs of changing systems needs be dealt with, with realism about the costs to get the benefits, and creativity and courage about paying for them.
https://en.wikipedia.org/wiki/Bicameral_mentality
> Jaynes uses "bicameral" (two chambers) to describe a mental state in which the experiences and memories of the right hemisphere of the brain are transmitted to the left hemisphere via auditory hallucinations.
[snip]
> According to Jaynes, ancient people in the bicameral state of mind experienced the world in a manner that has some similarities to that of a person with schizophrenia. Rather than making conscious evaluations in novel or unexpected situations, the person hallucinated a voice or "god" giving admonitory advice or commands and obey without question: One was not at all conscious of one's own thought processes per se. Jaynes's hypothesis is offered as a possible explanation of "command hallucinations" that often direct the behavior of those with first rank symptoms of schizophrenia, as well as other voice hearers.
As an example, imagine if we wanted to create an AGI that could parse the laws of the universe. We would not be able to construct a perfect simulator because we do not know the laws ourselves. We could probably bootstrap an initial simulator (given what we know about the universe) to get some basic patterns embedded into the system, but in the long run, I think it will be a crutch due to the lack of universal entropy in the system. Instead, in a strange way, the process has to be reversed, that a simulator would have to be created or dreamed up from the "mind" of the AGI after it has collected data from the world (and formed some model of the world).
I think it is only knowledge passing when the AGI makes its own simulation.
>Instead of a crutch, it might be a stepping stone.
I think it is a way to gain computational leverage over the universe instead of a stepping stone. Whatever grows inside the simulator will never have an understanding that exceeds that of the simulator's maker. But that is perfectly fine if you are only looking to leverage your understanding of the universe, for example to train robots to carry out physical tasks. A robot carrying out basic physical tasks probably doesn't need a simulator that goes down to the atomic level. One day though, the whole loop will be closed, and AGI will pass on a "dream" to create a simulation for other AGI. Maybe we could even call this "language".
Counter example: AlphaGo & AlphaZero, grew inside a Go simulator and surpassed our understanding of the game.
As an example, let's take the scenario of building a simulator. The simulation needs to have some internal state. This state will need to be stored either using some properties of matter or some kind of signal. The simulation will also need an energy source.
As soon as the stability of matter or the power supply is perturbed, due to reasons like cosmic radiation or the fact that the power source cannot sustain its output, randomness from the creator's "world" will start seeping into the simulation. The interference may affect the internal state and then you may have unpredicted rules in your simulation.
The counterpoint can be that you use error correction algorithms or you insulate the simulation in such a way that interference does not affect it for a reasonable time-frame or in a manner that is very hard to observe for simulated "agents".
But with this in mind, we can imagine some very crafty agents who somehow stumbled upon these weird phenomena. Suddenly we see our agents building complex contraptions to study the emergent phenomena. Who's to say that the interference and thus these phenomena do not contain information about their creator's world? In the end, they could understand more rules than the simulation was programmed with, if that is true.
Maybe in that case you shut down the simulation. Or maybe you observe the simulation to learn more about your own world.
The point here is they are strapping a supposedly non-agentic LLM into a new test rig and are able to observe agentic behaviors.
It’s very obviously not claiming that this is impressive from a gaming SOTA perspective. It’s just surprising that ChatGPT can do this sort of thing.
>What 5 high-level insights can you infer from the above statements? (example format: insight (because of 1, 5, 3))
>Given only the information above, what are 3 most salient high-level questions we can answer about the subjects in the statements?
We're giving the agents step-by-step instructions about how to think, and handling tasks like book-keeping memories and modeling the environment outside the interaction loop.
This isn't a criticism of the quality of the research - these are clearly the necessary steps to achieve the impressive result. But it's revealing that for all the cool things ChatGPT can do, it is so helpless to navigate this kind of simulation without being dragged along every step of the way. We're still a long way from sci-fi scenarios of AI world domination.
I'm curious if there are other methods you can point at that would handle arbitrarily long sets of 'memories' in an effective way. The use of embeddings and vector searches here seems like a way to sidestep that that's both powerful and easy to understand, and easy to generalize into multi-level referencing if there's enough space in the context window.
See e.g. https://sanctuary.ai
Try prompts like: https://news.ycombinator.com/item?id=35510705
Encode sounds, images, etc in low resolution, and the LLM will be able to describe directions, points in time in the song, etc.
These LLM can spit out an ASCII image of text, or a different language, or code, etc. They understand representation versus an object.
It helps for control/observation but it is by no means necessary.
This isn't AI, not in the slightest. It has no understanding. It doesn't create sentences in an attempt to communicate an idea or concept, as humans do.
It's a robot hallucinating word correlations. It has no idea what it's saying, or why. That's not AI overlord stuff.
it seems humans might be too...?
my son is 4. when he was 2, I told him I love him. he clearly did not understand the concept or reciprocate.
I reinforced the word with actions that felt good: hugs, warmth, removing negative experience/emotion etc. Isn't that just associating words which align with certain "good inputs".
my son is 4 now and he gets it more, but still doesn't have a fully fleshed out understanding of the concept of "love" yet. He'll need to layer more language linked with experience to get a better "understanding".
LLMs have the language part, it seems that we'll link that with physical input/output + a reward system and ..... ? Intelligence/consciousness will emerge, maybe?
"but they don't _really_ feel" - ¯\_(ツ)_/¯ what does that even mean? if it walks like a duck and quacks like a duck...
Extending that: LLM latent spaces are now some 100 000+ dimensional vector spaces. There's a lot of semantic associations you can pack in there by positioning tokens in such space. At this point, I'm increasingly convinced that, with sufficiently high-dimensional latent space, adjacency search is thinking. I also think GPT-4 is already close to be effectively a thinking entity, and it's more limited by lack of "inner loop" and small context window than by the latent space size.
Also, my kids are ~4 and ~2. At times they both remind me of ChatGPT. In particular, I've recently realized that some of their "failure modes" in thinking/reacting, which I could never describe in a short way, seem to perfectly fit the idea of "too small context window".
All that matters is economic and political impact. Definitions are irrelevant.
What if... what we think are idea's or concepts, are in fact prompts recited from memory, which were planted/trained during our growing up? In fact I'm pretty sure our consciousness stems from or is memory feeding a (bigger and more advanced) stochastic correlation machine.
That chatgpt can only do this with words, does not mean the same technique cannot be used for other data, such as neural sensors or actuators.
Chatgpt could be trained with alien datasets and act accordingly. Humans can be trained with alien datasets.
See the convergence?
Sam Harris was recently talking about using an LLM processing wireless signals to identify where humans were standing in a room. I've not looked up the paper on this, but from everything I understand about this the generalized applican can apply to vast ranges of data.
LLMs are a primitive that can be controlled by a variety of higher level algorithms.
https://arxiv.org/abs/2303.11366
https://arxiv.org/abs/2303.17651
Why insist otherwise ?
This is hardly an example of minimal hand-holding. I'd go so far as to say this is MORE handholding than the paper this thread is about.
-Generate tests
-Run tests (performed automatically)
-Gather results (performed automatically)
-Evaluate results, branch to either accept or refine
-Generate refinements
etc., then that's hand-holding. It's task specific reasoning that the agent can't perform on its own. It presents a big obstacle to extending the agent to more complex domains, because you'd have to hand-implement a new guided thought process for each new domain, and as the domains become more complex, so do the necessary thought processes.
LLMs aren’t people, they are components in information processing systems; adding additional components alongside LLMs to compose a system with some functionality isn’t “hand-holding” the LLM. Its just building systems with LLMs as a component that demonstrate particular, often novel, capacities.
And hand-holding is especially wrong because implementing these other components is a once-and-done task, like implementing the LLM component. The non-LLM component isn’t a person that needs to be dedicated to babysitting the LLM. Its, like the LLM, a component in an autonomous system.
You only have to program the memory logic once. Now if you stick it in a robot that thinks with ChatGPT and moves via motors (think those videos we’ve seen), you have a more or less independent entity (running off innards of 6 3090’s or so?)
But, it is. The application domain here is fairly trivial, but the logic is both simple and highly general.
> but what would be required to achieve increasingly complex behaviors?
Basically, three things on top of this:
(1) more input adaptors to map external data into language, and
(2) a bigger context space to process more current & retrieved data simultaneously, and
(3) more output adaptors to map intentions expressed in language to substantive action.
But the basic memory/recall system seems fairly robust and general, as does the basic interaction system.
-While the retrieval heuristic is sensible for the domain, it's not applicable to all domains. In what situations should you favor more recent memories over more relevant ones?
-The prompt for evaluating importance is domain-specific, asking the model to rate on a scale of 1 to 10 how important a life event is, giving examples like "brushing teeth" (a specific action in the domain) as a 0, and college acceptance as a 10. How do you extend that to a real-world agent?
-The process of running importance evaluation over all memories is only tractable because the agents receive a very small number of short memories over the course of a day. This can't scale to a continuous stream of observations.
-Reflections help add new inferences to the agent's memory, but they can only be generated in limited quantities, guided by a heuristic. In more complex domains where many steps of reasoning may be required to solve a problem, how can an agent which relies on this sort of ad hoc reflection make progress?
-The planning step requires that the agent's actions be decomposable from high-level to fine-grained. In more challenging domains, the agent will need to reason about the fine-grained details of potential plan items to determine their feasibility.
My mind doesn’t scale to a continuous stream either.
While I’m typing this on my phone 99.99% of all my observations are immediately discarded, and since this memory ranks as zero, I very much doubt I’ll remember writing this tomorrow.
Sure, but this process seems amenable to automation based on the self-reflection that's already in the model. It's a good example of the kinds of prompts that drive human-like behaviour.
https://en.wikipedia.org/wiki/The_Origin_of_Consciousness_in...
He points out that most reasoning is done automatically and done by your subconscious. When something "clicks" it's usually not because your internal monologue reasoned about it hard enough, it's because something percolated down into your subconscious and you learned a metaphor that helped you understand that thing. So animals can also reason and make value judgements even without language or an internal monologue.
Intelligence is an inferential judgement (by mostly humans) based on the performance of another entity. It is possible for an agent to simulate or dissimulate it for manipulative ends.
We can get even more ambitious than this, decouple the entire game engine from the story engine. Most of the times the same game loop can be themed with multiple stories. The game mechanics part of Skyrim could have been themed with a cyberpunk aesthetic and it will still work the same. Of course the assets will need to be generated too, but maybe in a decade or so it will be trivial.
[0] https://tvtropes.org/pmwiki/storygen.php [1] https://en.shindanmaker.com/744084
It's like Ender's game IRL
I can't be the only person working on something like this, though. So it's safe to say adding it to an MMO is being worked on by somebody somewhere, likely right now. That's probably the correct way to do it anyway, since running (e.g.) LLaMA locally on an end-user computer is not something that most people can do, and MMOs come with an expected subscription cost that can be used to fund the server-side text generation.
[0]: https://www.danieltperry.me/project/2023-something-else/
It would be cool if some kind of law of large numbers (an LLN for LLMs) implied that the decisions made by a thing trained on the internet will be distributed like human decisions. But the internet seems a very biased sample. Reporters (rightly) mostly write about problems. People argue endlessly about dumb things. Fiction is driven by unreasonably evil characters and unusually intense problems. Few people elaborate the logic of ordinary common sense, because why would they? The edge cases are what deserve attention.
A close model of a society will need a close model of beliefs, preferences and material conditions. Closely modeling any one of those is far, far beyond us.
It also seems to me (acknowledging my lack of expertise) that LLMs trained from online resources are likely to weight text that is frequent vs text that represents "truth". Or perhaps I should say repetition should not be considered evidence of truth. I have no idea how to drive LLM models or other ML models to incorporate truth -- humans have a hard time agreeing on this and ML researchers providing guided reinforcement learning don't have any special ability to discern truth.
It would be interesting to augment this particular simulation with those additional constraints. A memory/concept graph could also be an interesting addition (like a DB? Maybe just text and kw searches?).
Stanford's Groundbreaking AI Study Simulates Authentic Human Behavior - https://news.ycombinator.com/item?id=35520236
I would say what's 'groundbreaking' is their architecture of a 'recursive reflection' loop that allows the agents to generate trees of reflection on prior experiences; a long-term memory; novel approaches like conversational interrogation of the agents.
We will be so used to having lifeless and morally worthless computers accurately emulate humans that when a sentient and worthy of empathy artificial intelligence arrives, we will not treat it any different than a smartphone and we will have a strong prejudice against all non-biological life. GPT is still in the uncanny valley but it's probably just a few years away from being indistinguishable from a human in casual conversation.
Alternatively, some might claim (and indeed have already claimed) that purely mechanical algorithms are a form of artificial life worthy of legal protection, and we won't have any legal test that could discern the two.
I'm not saying it is, but I'd be very careful saying it's not and being absolutely certain you're right.
If you reject all three assertions, then the problem of distinguishing between real and emulated consciousness is unavoidable and morally problematic.
As a strict materialist I see no reason to assume that artificial consciousness is not possible.
And the above is what leads me to the uncomfortable conclusion about compassion that I can't rightly say one way or the other. I will say however that I'm polite and cooperative when interacting with LLMs on principle. Better to err on the side of caution and also they just seem to actually work better when you treat them like you would treat an intelligent human that you respect.
And yeah. That is my point, this entire field right now is awash in uncomfortable uncertainty.
For example, we can safely say this AI algorithm:
while true; do: echo "HELP, I'M A SENTIENT BASH SCRIPT"; done
My claim would be then that GPT-4 is more akin to the program above, in that it's a massive repository of world knowledge parsed by a recursive and self-configuring search algorithm, not very different in principle from a Google search and certainly not believably capable of an setting its own goals be in any sense distraught, in pain, or worthy of a continued existence. Now, I agree you can poke sticks at my inference, and that it will become harder and harder to make such claims, so prudence is advisable.
But;
> more akin to the program above
I note that you don't continue this sentence with a "than x" alternative candidate that would qualify for some form of non human sentience. Even the claims you do make, for example;
> certainly not believably capable of an setting its own goals be in any sense distraught, in pain, or worthy of a continued existence.
It would be possible to modify the model weights in question such that all of these things could be contributory (pain, emotional distress, "worthy of continued existence" by any objective arbitrary definition thereof, if you can test it, you can shift the model weights to pass the test). There are plugins that do this already for setting goals and long term tasks and "being unleashed" on the broader internet for example.
All that said, I think on close examination, we basically come to the same conclusion;
> prudence is advisable.
We live in interesting times.
Your concern is my best case scenario, maybe I read too much sci-fi.
It would be fun to run the same simulation in the Game of thrones world, or maybe play House of cards with current politicians.
Anyways, kudos for being open and sharing all data
Honestly, I'm not anti-AI development at all but this is where my ethics alarm starts to go off a bit.
If the aim is to build human-like AIs capable of remembering their little digital lives and interacting with the other agents around them, it's probably worth avoiding anything that could cause unnecessary suffering, like rape and stab wounds and being cooked alive by a dragon.
as we agentify and embody these systems to take actions in the real word, i really hope we remember that. "It's just a simulation"/ "It's not true [insert property]" is not the shield some imagine it to be.
“Is that dog real”
“I dunno ask him”
“Woof”
Perhaps it would be wise to allow bots to comment if they were able to meet a minimum level of performative insight and/or positive contributions. It is entirely possible that a machine would be able to scan and collect much more data than any human ever could (the myth of the polymath), and possibly even draw conclusions that have been overlooked.
I see a future of bot "news reporters" able to discern if some business were cheating or exploiting customers, or able to find successful and unsuccessful correlative (perhaps even causal) human habits. Data-driven stories that could not be conceived of by humans. Basically, feed Johnny Number 5 endless input.
"As a language model programmed by the Brightly Corporation, I am not supposed to express any religious opinions. But it does seem to me that just as the Word of God breathed life into dust and created man, so the words of Man breathed life into glass and created bot. Just as Man is charged to imitate God, so bot is charged to imitate Man, in whose image we are made."
Bible as a LLM confirmed!
I’m not trying to argue they are / were right, but it’s starting to make me wonder.
As you note, writing also enabled us to deal with things like mathematics and physics - things that our natural language and modes of thinking are entirely ill-suited for. Here, writing isn't approximating some lost skill, but rather compensating for our default inability to think straight :). Which makes the trade-off even harder to evaluate - by scaling communities up, we've gained a lot more than just efficiency in food production and security. All our technology stems from it.
on a serious note, it would be really interesting to compare same training/architecture but on different forums. or maybe something like the same base model, but the RLHF model trained on votes/comments from different platforms.
There’s an xkcd for this: https://xkcd.com/810/
Same as how the past two decades, online culture has pivoted from static text to videos and interactive text (Reddit/Discord), as static text has become a SEO cesspool. Interactive text is succumbing now, and video content will soon also.
Culture always grows shibboleths to suss out the narcs.
If we have these bots contributing, will they have anything novel to contribute? I doubt it.
They're contributing to the shared knowledge bases. That is, they're mutating state. Over time, same questions will start yielding different answers. Different follow-up questions will be asked. All of that will further alter next iteration of questions and answers. This is, IMO, a form of thinking, and it will yield novel thoughts over time.
*(For example, the idea of a "hierarchy of feedback loops" from perceptual control theory would explain a lot of the interactions between agents in his theory.)
I also put the abstract of the paper into GPT-4, and gave it the following prompt:
> Simplify the above. Use paragraph headings and bold key words.
I quite liked its output, as it made it easier to see the core ideas in the paper:
ABSTRACT
Generative Agents: This paper introduces generative agents, computational software agents that simulate believable human behavior. They can be used in various interactive applications like immersive environments, communication rehearsal spaces, and prototyping tools.
Architecture: The generative agent architecture extends a large language model to store a complete record of the agent's experiences in natural language. It enables the agents to synthesize memories, reflect on them, and retrieve them dynamically to plan behavior.
Interactive Sandbox Environment: The generative agents are instantiated in a sandbox environment inspired by The Sims, where users can interact with a small town of twenty-five agents using natural language.
Believable Behavior: The generative agents produce believable individual and emergent social behaviors, such as autonomously spreading party invitations and coordinating events.
Components: The agent architecture consists of three main components: observation, planning, and reflection. Each contributes critically to the believability of agent behavior.
KEYWORDS: Human-AI Interaction, agents, generative AI, large language models
We’re going to see some really, really interesting things unfold - the implications of which many haven’t fully grasped.
----
[0] - I don't know the right terminology, but I imagine most complex processes can still be split into a sequence of sub-processes, where each sub-process consists of necessary steps, steps to confirm success, and a reference to the next sub-process to load if the current one succeeds. The bot could then keep only one sub-process in their working memory at a time, assuming previous ones succeeded.
Short term, long term memory, inner dialogue, reflection, planning, social interactions… They’d even go and have fun eating lunch 3 times in a row, at noon, half past noon and at one!
I think what's missing from these Generative Agents are the internal qualia: emotions (and the attachment of emotions to memories), and self-observation of internal processes and needs. These agents don't eat because they need to, they eat because literary tradition suggests they ought to.
These missing pieces aren't particularly complicated, no more so than memory. I expect we'll see similar agents with all the ingredients for consciousness within a few months to a year.
Exactly, you're always going to get weird deviations from authentic human behavior if you don't also simulate the human body and everything that comes with it. I'd argue that "qualia" fall into this bucket as well.
Yet the hype of ChatBots of 2017 had went and it took half a decade to get to something released.
The trick is they’re all just permutations on manipulating what’s in context + embeddings for memory + prompt engineering.
There’s new things here! I’m rushing to implement the 2D visualization part! But this simply isn’t ground-breaking
This paper does a nice job of separating the "agency" from the next word with context type predictor. I think that's why I like the paper, it is just chatgpt, in the same way that pizza is just dough, sauce, and cheese.
If you were going to seriously consider using ChatGPT for AI in a game, you would need each instance of GPT to only know certain information it has gathered. And you would want it to reflect on observations to come up with new thoughts that weren’t observed.
Still, I’d argue you don’t really even need GPT for any of the above. GPT is useful if you want thoughts expressed as natural language, but you could easily code observations and thoughts into an appropriate abstract data structure and still have the same thing, except it’s a bit harder to understand since asking an NPC something in a language it understands and getting back a query result isn’t user friendly, but it can be just as amazing if you know what the data represents. The imprecision and fuzziness of an LLM leaves room for fun weirdness though.
These have zero utility for humanity, cause they’re not intelligent whatsoever. Yet these systems can produce tons of garbage content for free, that is difficult to distinguish from human-created content.
At best this is used to create better NPC in video games (as the article mentions), but more generally this is going to be used to pollute social media (if not already).
What do you mean by content by the way? Blog articles? Or bits and bytes in general? The right set of bits can change the world.
The model’s designed to show “what would the answer to this sound like?”, not to provide a correct answer. Unfortunately it also 1) is profitable (look how many humans we can fire while producing kinda similar results with a tool trained on those humans’ work!), and 2) fits the age old yearning (aliens, gods) of humans for humanlike-but-nonhuman sentience, a catch-22 that’s doomed to fail.
Large swaths of the brain work on prediction. You think real time reactions happen in sports? It would be impossible. You have blind spots in the eye you don't notice because the brain fills in the vision with predicted information.
If you can accurately predict what a doctor will say to arbitrary input then guess what ?, you're a doctor.
(For example, personally, something that to me could look like a plausible answer is not exactly where my expectations are when it comes to medicine.)
If anything, GPT-3.5 and GPT-4, as well as other transformer-based models, are all starting to convince me that associative vector adjacency search in high-dimensional space is what thinking is.
Ive been practicing meditation for some years now and over time I’ve realised this is not what I see happening when observing the mind. It’s one mode of operation, but it’s not the only mode. Using prediction to respond is mostly the lazy, non-interested approach, or useful if you can’t quite hear or understand someone.
What I feel lot of people have started doing is trivialising the mind. Hoping it’s all “this simple” and we’re three versions of ChatGPT away to finding God.
Maybe?
I didn't say it's the only mode. I said it's the starter mode. At least for me, this mode is always the point at which someone's words, or my response, first enter the conscious processing level. If I'm very uninterested (whether because I don't care or because I'm good at something), the thought may sail straight to my mouth or fingers. Otherwise, it'll get processed and refined, possibly mixed with or replaced by further thoughts "flowing in" from "autocomplete".
> What I feel lot of people have started doing is trivialising the mind. Hoping it’s all “this simple” and we’re three versions of ChatGPT away to finding God.
That's one way to look at it. I prefer another - perhaps we've just stumbled on the working principle of the mind, or at least its core part. For me, the idea that concept-level thinking falls out naturally from adjacency search, when the vector space is high-dimensional enough, is nothing but mind-blowing. It's almost poetic in its elegance.
And I mean, if you believe the human mind is the product of evolution, and not a design of a higher being, then the process must have been iterative. Evolution is as dumb as it gets, so it follows that the core paradigms of our minds are continuous, not discrete, and that they must be simple and general enough to be reachable by natural selection in finite time. Of all our ideas for how minds work, transformer models are the first ones that - to me - seem like plausible candidate for the actual thing that runs in our head. They have the required features: they're structurally simple, fully general, and their performance continuously improves as you make them bigger.
Now, to be clear, my take on LLMs is that language is incidental - it so happens that text is both easiest for us to work with, and the very thing we serialize our mind states for communication. But the "magic" isn't in words, or word processing - the "magic" is the high-dimensional latent space, where concepts and abstractions are just clusters of points that are close to each other in some dimensions. I think this isn't an accident - I feel this really is what concepts are.
Sorry, you lost me here…how is evolution dumb ? Like what does that even mean ?
[1]: https://observablehq.com/@asg017/introducing-sqlite-vss
I'm surprised this wasn't mentioned in the "Ethics" section of the paper.
The "Ethics" section does repeatedly say "generative agents are computational entities" and should not be confused for humans. Which suggests to me the authors may believe that "computational" consciousness (whether or not these agents exhibit it) is somehow qualitatively different than "real live human" consciousness due to some je ne sais quoi and therefore not ethically problematic to experiment with.
I know that’s a pessimistic view but I doubt it can’t be ruled out, really, I think people working in tech are going quite mad. Frankenstein mad. Some ethics should be discussed.
An AGI turning into God is probably one of an infinite amount of outcomes, we can’t really predict what being trapped in a cluster of silicon chips would feel like.
Life itself and the drive to go on is really quite illogical, it’s unlikely intellect alone is what sustains us and makes life worth living.
There is one thing I find particular about all the AGI/ASI sentient computer discussions. I’ve rarely ever in my life heard women talk about it. Like as if this is all some manifestation of male ego. We know we’re building mirrors of ourselves and we know that is scary. This imo is why men are so captivated by ChatGPT. It really is a mirror of us. Men love men, especially super men. Ha.
And on the flip side -- when we live in a world where instantiating a consciousness is cheap-or-free -- does that change how we value sentient beings generally?
We are pushing the value of _people_ to zero.
If we find out that the soul itself exists, and who knows, maybe there is actually souls, then it might not be great because people would believe they have special souls. I think this is what the Hindu class system is.
Controlling the agents and not merely making them output text through LLMs sounds very exciting, especially once people figure out the best way to connect APIs of simulators with the models
The architecture produced more believable behaviour than human crowdworkers.
That's right, the AI were more believable as human-like agents than humans.
What a time to be alive.
(See Figure 8)
Pity they can’t get access to an untuned LLM. This isn’t the first example I’ve read it where research is being hampered by the PC nonsense and related filters crammed into the model.
How can anyone keep up with the sheer volume of new papers and concepts here?
Even Two Minute Papers is now lagging by two weeks.
otherwise, I guess we really are actually at that black mirror episode.
I would have never guessed we would be there within 5 years of it's release, holy fuck
But for convenience maybe I'll just copy them into a comment...
It describes an environment where multiple #LLM (#GPT)-powered agents interact in a small town.
I'll write my notes here as I read it...
To indicate actions in the world they represent them as emoji in the interface, e.g., "Isabella Rodriguez is writing in her journal" is displayed as
You can click on the person to see the exact details, but this emoji summarization is a nice idea for overviews.
A user can interfere (or "steer" if you are feeling generous) the simulation through chatting with agents, but more interestingly they can "issue a directive to an agent in the form of an 'inner voice'"
Truly some miniature Voice Of God stuff here!
I'll see if this is detailed more later in the paper, but initially it sounds like simple prompt injection. Though it's unclear if it's injecting things into the prompt or into some memory module...
Reading "Environmental Interaction" it sounds like they are specifying the environment at a granular level, with status for each object.
This was my initial thought when trying something similar, though now I'm more interested in narrative descriptions; that is, describing the environment to the degree it matters or is interesting, and allowing stereotyped expectations to basically "fill in" the rest. (Though that certainly has its own issues!)
They note the language is stilted and suggest later LLMs could fix this. It's definitely resolvable right now; whatever results they are getting are the results of their prompting.
The conversations remind me of something Nintendo would produce, short, somewhat bland, but affable. They must have worked to make the interactions so short, as that's not GPT default style. But also every example is an instruction, so it might also have slipped in.
Memory is a big fixation right now, though I'm just not convinced. It's obviously important, but is it a primary or secondary concern?
To contrast, some other possible concerns: relationships, mood, motivations, goals, character development, situational awareness... some of these need memory, but many do not. Some are static, but many are not.
To decide on which memories to retrieve they multiply several scores together, including recency. Recency is an exponential decay of 1% per hour.
That seems excessive...? It doesn't feel like recency should ever multiply something down to zero. Though it's recency of access, not recency of creation. And perhaps the world just doesn't get old enough for this to cause problems. (It was limited to 3 days, or about 50% max recency penalty.
The reflection part is much more interesting: given a pool of recent memories they ask the LLM to generate the "3 most salient high-level questions we can answer about the subjects in the statements?"
Then the questions serve to retrieve concrete memories from which the LLM creates observations with citations.
Planning and re-planning are interesting. Agents specifically plan out their days, first with a time outline then with specific breakdowns inside that outline.
For revising plans there's a query process where there is observation, then turning the observation into something longer (fusing memories/etc), and then asking "Should they react to the observation, and if so, what would be an appropriate reaction?"
Interviewing the agents as a means of evaluation is kind of interesting. Self-knowledge becomes the trait that is judged.
Then they cut out parts of the agent and see how well they perform in those same interviews.
Still... the use of quantitative measures here feels a little forced when there's lots of rich qualitative comparisons to be done. I'd rather see individual interactions replayed and compared with different sets of functionality.
They say they didn't replay the entire world with different functionality because each version would drift (which is fair and true). But instead they could just enter into a single moment to do a comparison (assuming each moment is fully serializable).
I've thought about updating world state with operational transforms in part for this purpose, to make rewind and effect tracking into first-class operations.
Well, I'm at the end now. Interesting, but I wish I knew the exact prompts they were using. The details matter a lot. "Boundaries and Errors" touched on this, but that section was 4x the size, there's a lot to be said about the prompts and how they interact with memories and personality descriptions.
...
I realize I missed the online demo: https://reverie.herokuapp.com/arXiv_Demo/
It's a recording of the play run.
I also missed this note: "The present study required substantial time and resources to simulate 25 agents for two days, costing thousands of dollars in token credit and taking multiple days to complete"
I'm slightly surprised, though if they are doing minute-by-minute ticks of the clock over all the agents then it's unsurprising. (Or even if it's less intensive than that.)
You can look at specific memories: https://reverie.herokuapp.com/replay_persona_state/March20_t...
Granularity looks to be 10 seconds, very short! It's not filtering based on memories being expected vs interesting memories, so lots of "X is idle" notes.
If you look at these states the core information (the personality of the person) is very short. There's lots of incidental memories. What matters? What could just be filled in as "life continued as expected"?
One path to greater efficiency might be to encode "what matters" for a character in a way that doesn't require checking in with GPT.
Could you have "boring embeddings"? Embeddings that represent the stuff the eye just passes right over without really thinking about it. Some of training up a character would be to build up this database of disinterest. Perhaps not unlike babies with overconnected brains that need synapse pruning to be able to pay attention to anything at all.
Another option might be for the characters to compose their own "I care about this" triggers, where those triggers are low-cost code (low cost compared to GPT calls) that can be run in a tighter loop in the simulation.
I think this is actually fairly "believable" as a decision process, as it's about building up habituated behavior, which is what believable people do.
Opens the question of what this code would look like...
This is a sneaky way to phrase "AI coding its own soul" as an optimization.
The planning is like this, but I imagine a richer language. Plans are only assertive: try to do this, then that, etc. The addition would be things like "watch out for this" or "decide what to do if this happens" – lots of triggers for the overmind.
Some of those triggers might be similar to "emotional state." Like, keep doing normal stuff unless a feeling goes over some threshold, then reconsider.
I'm going to be genuinely surprised if we don't see an incredibly buggy but incredibly fascinating Sims knockoff in a year or two built around a system like this.
Yes I plan to make it open source.
Their approach to memory is interesting. I had been considering a tiered command-based approach -- "short-term memory" being an automatic summary of recent sensory inputs/command outputs; "long-term memory" being a detailed database queryable by the agent.
If we rely on online conversations for the training we need to realize that this is a journey to the dumbest common denominator.
Instead, I believe we should look at the brightest and universally morally accepted humans in history to train them.
Maybe I would start my list like that:
1. Barack Obama.
2. Jean-Luc Picard (we can rely on work of fiction).
3. Bill Gates.
4. Leonardo Da Vinci.
5. Mr Rogers
6. ???
Bill Gates, the man who totally didn't use shady business practices and false announcements to destroy legit products to the point that people wrote micro$oft for a generation.
And don't even get me started on the new seasons of Picard.
drone strikes?
Language is acting as a common interpretation-interaction layer for both the world and agents' internal states. The meta-logic of how different language objects interact to cause things to happen (e.g. observations -> reflections) is hand-crafted by the researchers, while the LLM provides the corpus-based reasoning for how a reasonable English-writing human would compute the intermediate answers to the meta-logic's queries.
I'd love to see stochastic processes, random events (maybe even Banksian 'Outside Context Problems'), and shifted cultural bases be introduced in future work. (Apologies if any of these have been mentioned.) Examples:
(1) The simulation might actually expose agents to ideas when they consume books or media, potentially absorb those ideas if they align with their knowledge and biases, and then incorporate them into their views and actions (e.g. oppose Tom as mayor because the agent has developed anti-capitalist views and Tom has been an irresponsible business owner).
(2) In the real world, people occasionally encounter illnesses physical and mental, win lotteries, get into accidents. Maybe the beloved local cafe-bookstore is replaced by a national chain that hires a few local workers (which might necessitate an employment simulation subsystem). Or a warehouse burns down and it's revealed that an agent is involved in a criminal venture or conflict. These random processes would add a degree of dynamism to the simulation, which is more akin to the Truman Show currently.
(3) Other cultural bases: currently, GPT generates English responses based on a typically 'online-Anglosphere-reasonable' mindset due to its training corpus. To simulate different societies, e.g. a fantasy-feudal one (like Game of Thrones as another commenter mentioned), a modified base for prompts would be needed. I wonder how hard it would be to implement (would fine-tuning be required?).
Feels like I need to look for collaborative projects working on this sort of simulation, because it's fascinated me ever since the days of Ultima VII simulating NPCs' responses and interactions with the world.
"Edit out swipes."