HyperAgents: Self-referential self-improving agents
github.com
github.com
This, IMO is the biggest insight into where we're at and where we're going:
> Because both evaluation and self-modification are coding tasks, gains in coding ability can translate into gains in self-improvement ability.
There's a thing that I've noticed early into LLMs: once they unlock one capability, you can use that capability to compose stuff and improve on other, related or not, capabilities. For example "reflexion" goes into coding - hey, this didn't work, let me try ... Then "tools". Then "reflxion" + "tools". And so on.
You can get workflows that have individual parts that aren't so precise become better by composing them, and letting one component influence the other. Like e2e coding gets better by checking with "gof" tools (linters, compilers, etc). Then it gets even better by adding a coding review stage. Then it gets even better by adding a static analysis phase.
Now we're seeing this all converge on "self improving" by combining "improving" components. And so on. This is really cool.
Another thing that get quantized is video preferences to maximize engagement.
OTOH, there's loads you can do for evaluation before a human even sees the artifact. Things like does the site load, does it behave the same, did anything major change on the happy path, etc etc. There's a recent-ish paper where instead of classic "LLM as a judge" they used LLMs to come up with rubrics, and other instances check original prompt + rubrics on a binary scale. Saw improvements in a lot of evaluations.
Then there's "evaluate by having an agent do it" for any documentation tracking. Say you have a project, you implement a feature, and document the changes. Then you can have an agent take that documentation and "try it out". Should give you much faster feedback loops.
I asked Claude to build a web app to run locally polling data from the LAN. It fought me for four rounds of me telling it that the data from the api wasn’t rendered on the page. It created tests with mock data, it validated the api, it tested that the page loaded. It was gaslighting telling me that everything worked every time I told it that it didn’t work. I had to tell it to inspect the dom and take screenshots with Playwright to make it stop effing around. I don’t think it ever would have found the right response on its own.
Even after deliberate intervention, it regressed a few rounds later and stopped caring that tests failed. Whatever, I don’t treat it as anything more than a sometimes-correct random output machine.
Larger composition, though, starts to run into typical software design problems, like dependency graphs, shared state, how to upgrade, etc.
I've been working on this front for over two years now too: https://github.com/smartcomputer-ai/agent-os/
I've been seeing the same thing. Where agents are great solving the immediate task, but as changes compound they run into software & architectural design problems. I created https://github.com/andonimichael/arxitect to help at least have coding agents self reflect on their software design. But I really like your approach to self-modification and improving the agent itself instead of just teaching it another skill in it's context.
> once they unlock one capability,
What does it mean to unlock? Its an llm nothing is locked. The output is a as good as the context, model and environment. Nothing is hidden or locked.
1. Coding is, as a technical endeavour, relatively difficult (similarly for mathematics). So a model which performs well on this task can be expected to easily handle also-technical-but-slightly-easier tasks, like understanding (musical) harmony theory or counterpoint -- for much the same reason that human programmers/mathematicians/scientist don't struggle to understand those "easier" theories.
2. Reinforcement learning augments a base models ability to excel in something else that's "difficult", namely to "look ahead" and plan multiple steps in advance. That's literally how the training algorithm works, generating multiple paths at once, and rewarding intermediate steps in those paths which succeed in attaining the goal. And that skill, too, is extremely useful in other domains. An AI agent which learns that to break a problem into sub-problems, and then tackle each in turn methodically -- it stands to reason that it can apply that to, say, a business plan.
Note: 1 & 2 are not independent, nor are frontier models' excellence in these domains magical: it ultimately boils down to the availability of massive datasets (in particular for coding) and totally objective metrics (in the case of mathematics: solved math problems). That's the key ingrediant for reinforcement learning to be so effective.
So: the skills are transferrable because they're difficult, and require lots of planning. That models are so good at them is a fluke, and in a parallel world where humans created git repo after git repo of business plans, it might be that which we lean on to teach a reinforcement learning algorithm how to "reason" and "plan".
Now let's turn our attention to the "synergies" aspect, which I agree with. Let's say your agentic model, which is already excellent at reasoning and planning, acquires a new or improved capability which allows it to search the domain space, calculate, etc. much better than before -- this capability can now bear upon the plan, or be factored into the plan. For example, the model might be able to say "I don't need to worry about this particular subproblem for now; I can rely on my "mathematica" capability to deal with it when I absolutely need."
Or to put it differently: monkeys, like humans, are able to use (rudimentary) tools. They'll take a rock, and use it to crack open a coconut (or whatever). But a human being, with far superior reasoning and planning abilities, takes that tool, and uses it to make an even better tool -- and the result after many iterations of this process is civilization as we know it, while monkeys are still stuck trying to crack open nuts with rocks.
This is the exact point I make whenever people say LLMs aren't deterministic and therefore not useful.
Yes, they are "stochastic". But you can use them to write deterministic tools that create machine readable output that the LLM can use. As you mention, you keep building more of these tools and tying them together and then you have a deterministic "network" of "lego blocks" that you can run repeatably.
Or, are there two. One is modifying the other, observing results, before self applying.
It is an extension of their DGM paper. Also it's ~88M+ tokens per full run I think, not surprising as any sort of exploratory search is expensive and I commend them for releasing the code online because it pushes this small subfield. But people need to temper their expectations. IMO the best part is a nice transfer between improvement objectives after exhaustive iteration that they found. I am wondering if what we have here is a way to exhaust local search space, by letting the model better express it.
On a separate one thing I think a lot about is whether these unchecked hyped claims and terms and marketing of papers actually does more bad than good to the field by setting expectations that cannot be delivered and distracting from the actual hard and unsexy nature of problems that need to be solved.
Its is the core of any and all learning/exellency; exposure to chaotic perturbations allow selection of solutions that are then generalized to further, ever more straining problems; producing increasingly applicable solutions.
This is the core of evolution, and is actually derivable from just a single rule.
I've always felt that the most important part of engineering was feedback loops.
Maybe nature is the greatest engineer ever?
The great oxygenation was a time of near catyclismsic scarcity for most complex organisms, as resources scale to food/energy requirements imply the most complex organisms were the most dependent on the environment, and were most impacted by changes.
Inversely, oxygenation was our most crucial abundancy pre cursor, as it provides a large substrate chemically for life to exhibit
On this view, learning in general operates via selection under uncertainty. This is less visible in individual cognition, where we tend to over-attribute agency, but it is explicit in science: hypotheses are proposed, subjected to tests, and selectively retained, precisely because the future cannot be deduced from the present.
In that sense, generation/discrimination is a particular implementation of this broader principle (a way of instantiating variation and selection) not the primitive itself.
Variation (chaos) comes from the tidal push/pull of all cumulative processes - all processes are nearly periodic (2nd law) and get slower - guaranteeing oscillator harmonics at intervals.
These intervals are astronomically convulted, but still promise a Fourier distribution of frequency: tidal effects ensure synchronization eventually, as all periods resonate eventually.
As systems are increasingly exposed to pendulums of positive and negative coherence, they will generalize for variance, and eventually for increasingly (fourier) selective filters of increasingly resiliente traits, that will generalize.
The system would eventually be increasingly resilient and eventually an awareness would develop.
Awareness of past periodic cycles would improve fitness (with or without consciousness) and eventually the mechanistic processes would be in the systems nature.
This is why we have pointless traditions, folk lore, collective unconscious artifacts, cyclical cataclysmic religions, the Fermi Paradox, the great filters...
Variation and selection are woven, but understanding how it all stems from gravity by means of nearly perioidic oscillators (spinning planets, tidal pools, celestial bodies) due to the conservation of angular momentum, due to the 3body problem.....that is what took a genius to reconcile
Basically a dice with a bajillion sides is getting rolled along an increasingly slim poker table, house winning eventually.
Time only goes one way, protons dont decay, energy is radiated unto the cosmic background hiss, until homogeneity is reached as CMB, and entrophy reaches 1.
I dont know where it comes from, but I know the shape it makes as it rolls by.
I am not sure how this is a necessary conclusion to the premises you provide.
As Briggs–Rauscher reactions can eventually lead to Belousov–Zhabotinsky reactions, the system can maintain homeostasis with its environment (and continuing to oscillate) by varying reactants in a loss minimizing fashion.
This loss minimizing would be done during scarcity to limp towards an abundance phase.
This is the mechanism that hypothetical tidal pools batteries would had exhibited to continue between periods of sunlight/darkness/acidity that eventually gets stratified as a resilency trait.
Loss minimizing to a few problems will generalize into abstraction, and a few solutions will develop.
These systems with more generalizable resilency traits will encounter increasingly varied selective sieves.
Systems that survive this seive will exhibit increasingly sophisticated, generalizable solutions to prevent loss of needed dependent reactions/resources.
These solutions must exert influence to be effective; influencing the environment for its own benefit.
As systems influence their environment, delineation of "self" and "environment" becomes a fundamental barrier.
The system would prefer itself, or be outcompeted by a similar system that does.
This layer of semi-life like material would form between sunlight and the oscillating reaction, and eventually envelope it, minimizing surface tension by means of a spherical cell like structure.
Small stuff runs off of loss minimizing at a force level for its mechanistic affect; from covalent bonds to cellular ion transport, the path of lesser resistance is the fundamental forces.
As systems become more complex, the minimizing is less directly attributable to the fundamental forces and becomes more of a Byzantine dependency/feedback network.
This byzantine labyrinth of interactions is called biology.
The delineation of self, the ego.
At the highest levels, geopolitics. At the human level, mate suppression. Lowest level, energy conservation.
All loss minimizing
"Awareness" isn't a well defined term and is often just a proxy for consciousness. But in as much as we can define it, it is one or both of experience and knowledge. You may (or may not be) aware of the hum some electronics in your house. At certain points in the day that hum is present in you attention, at other points it is absent from your attention. Sometimes you choose to bring previously unattended objects into your awareness, sometimes they are thrust there despite your will.
What is actually interesting about awareness, and one of the reasons it is a tricky subject, is that it isn't clearly related to agency. There are objects of your awareness that you do not act on, and you act with respect to objects that are provably not in your awareness.
There is also the question of the field within which these oscillations take place. Is it the electro-magnetic field? A quantum field? Which field are we talking about? If you are proposing some "principle of least action" in that field, can you describe it?
You seem to claim "loss minimization" and then hand wave the rest. But without descriptions of knowledge and experience it feels like you aren't actually saying anything except stating an opinion that reduces all knowledge and experience to loss minimization. That is an extraordinary claim and requires either extraordinary evidence or extraordinary reasoning.
Beneficial behavior for systems/organisms dont require consciousness - most do better with less, looking at western nihilism...
Consciousness is just knowledge / conscientiousness with a loop, some garbage collection, and a generally meta-reflective process of self-directed improvement by learning from mistakes and using what works.
Mesa/super selective processes then test these individual instances en masse.
Entities dont need true understanding, just approximation.
Rain dances may not work, but they still increase reproductive odds.
Lamarck was right, but for the wrong reasons, which is apt.
The giraffe that stretches its neck convincingly (or wears high heels) and the giraffe that is naturally tall converge over generations as the female giraffes cannot differentiate between the two.
As for the medium that the oscillations permeate, it is all of them, and they all have an effect on each other.
The flooding of the Nile, the lunar affliction on our hormones, astronomical bombardment,pest blooms, the chances of bring born on a Saturday, agricultural invention, the distinctions in blood types, etc: are all seemingly unrelated things had the celestial mechanisms orchestrating their (near) periodic nature not been known - until modernity.
Yet, before modernity, these (near) cyclic phenomena were known about, and described, just 'mis'attributed as supernatural / God of the Gaps.
("Panem-synchronicity")
The forces are all substrates for complexity. The underlying force, as Hawking told us, was gravity.
This is the hand-waving I am talking about right here. We've stepped up from some kind of self-reproducing wave-like patterns in some combination of mediums/fields and somehow moved passed "understanding" and onto its consequences.
Is your position that subjectivity is emergent or inherent? Either requires some kind of defense, either empirically or rationally.
Implying that we are a bunch of waves appreciating waves caused by gravity is poetic but it breaks down in the "appreciating" part. None of your examples of cyclic recurrence (river floods, moon cycles, etc.) lead to understanding, knowledge, experience or awareness.
This criticism doesn't suggest that rhythmic cycles, stability in wave-like systems, gravity, loss-minimization or any other part of your system are wrong. It suggests that there is something missing in either their combination or some additional missing piece.
There is a point, perhaps some region of a gradient/spectrum, where gravity and it's effect on wave-like fields manifests as subjectivity/understanding/experience/awareness. You are just hand waving past that implying it is "necessary". In that way, you are assuming rather than explaining.
I do think subjectivity is both, it emerges gradually as an increasingly sophisticated inner model of the entities' outter world; as an innate loop of feedback between senses, abilities, and environment.
I see attention itself as exhibiting wave-like structures. On one end there is the totally holistic, the systemic view in which there is an opening up of possibilities. On the other end is the totally reductionistic, the atomic view in which there is particular discernment between options. Attention seems to shift between these two modes, not as a naive binary but across a spectrum like we have been discussing.
In my view, neither view is privileged. In fact, I make an even more radical hypothesis that it is the motion between these polls that is the interesting feature of attention. At some points we focus on details, at other points we focus on systems - the defining feature is not the scope of attention at any particular time but its change over time.
What I note in your comments is a strong tendency towards either the massive systemic or the miniature atomic (the far extremes of the poles of attention). My push back has been towards the middle ground between them, towards the point at which awareness arises within your system. It is like you are zooming along a wave and slowing down at the top (universal scale) and bottom (atomic scale) but rushing through the mean (human scale).
Another way to say this, I don't believe either a systemic model nor a reductionist description will be adequate to describe how systems (or atoms) become lifelike. I believe the description will involve some oscillation between these perspectives, and that the most fruitful region to explore will be the region roughly at human scale, the place where we see the most advanced life.
However, your points about loss minimization and gravity are well worth further study and I thank you for bringing them to my attention.
While I disagree with your notion that this is explicity due to gravity, the rest of your argument seems to align with some of this lab's work. Learning can be demonstrated on scales as low as a few molecules, way below what we would normally call "life".
Gemini itself says AGI will be here in 2029, with human level intelligence and self-improvement capabilities. But then it will take until 2045 before the singularity. I don’t understand what they are going to do in all those years.
Wittgenstein said "From what we can not speak of, we must be silent about". That sounds like a tautology but I think there is a deeper meaning behind it.
It means simply that once you start talking about what we can not talk about, you are already talking about that and therefore it is NOT something you can NOT talk about. Clearly we can talk about it because we are already talking about it. And therefore it is not something that can not be talked about. That is a paradox, a bit like Godel's, but something that doesn't contradict itself.
That's exactly what the Singularity is: it's the transition point beyond which meaningful predictions aren't possible.
In a black hole it's the center where relativity breaks down.
In AI it's the point at which non-human intelligence no longer requires human intelligence for self improvement: after which predictions of the future become somewhat meaningless.
In the human lives experience, I would argue its like having your first child: you can know what's coming, study the theory, know everything to expect and youre still you on the other side...but you can't really know what will happen till you get there.
I think we are already in the stage where AI can and does improve itself. But why should this stage be called "Singularity"? Like a Black Hole? That sounds like hype to me.
When AI can improve itself, wouldn't it still be able to explain to us how it has improved itself? If it can not it still has a lot of improvement to do.
Or are we saying that some things are "unexplainable" and AI will discover such things without being able to explain to us what they are? That sounds like mysticism, or hype to me. Or religion. We can not explain God, right?
It is significant that if machine-intelligence can improve itself, then there's obviously a transition point at which the bulk of innovation shifts from being done by human minds to being done by machine minds - and if nothing else that's a "phase" transition in technological progress, namely that the future of technological development is no longer the product of human innovation but of our created machines, hopefully acting according to our desires.
The technological singularity is that: the pace of innovation separates from the ability of human minds to keep up with understanding it. We can barely predict our own innovation: we would be unable to usefully predict innovation happening by alien intelligences operating faster then we can think.
Now of course where that line exists, or if it's even plausible, are separate questions (there's a reasonable argument that physical manufacturing is likely to be a serious limiter for a good long time still at the very least).
IF we had a black box programing language, and handed it over to this system, it would never be able to do anything with it past its context window.
Hey kids I hear you like agents, so we made an agent write agents till we got better agents.
I was annoyed at how Claude Code ignores my CLAUDE.md and skills, so I was looking for ways to expand type checking to them. So I wrote a wrapper on top of claude-agents-sdk that reads my CLAUDE.md and skills, and compiles them into rules - could be linter rules or custom checking scripts. Then it hooks up to all tools and runs the checks. The self improving part comes if some rule doesn't work: I run the tool with the session id in review mode, it proposes the fixes and improves the rule checkers. (not the md files) So it's kinda like vibe coding rules, definitely lowers the bar for me to maintain them. Repo: https://github.com/chebykinn/agent-ruler
I wonder how much of the improvement is due to the agent actually learning new things vs. reaching parts of its latent space that enable it to recall things it already knows. Did the agent come up with novel RL reward design protocols based on trial and error? Or did the tokens in the environment cause it to "act smarter"?
Abstract:
Self-improving AI systems aim to reduce reliance on human engineering by learning to improve their own learning and problem-solving processes. Existing approaches to self-improvement rely on fixed, handcrafted meta-level mechanisms, fundamentally limiting how fast such systems can improve. The Darwin Gödel Machine (DGM) demonstrates open-ended self-improvement in coding by repeatedly generating and evaluating self-modified variants. Because both evaluation and self-modification are coding tasks, gains in coding ability can translate into gains in self-improvement ability. However, this alignment does not generally hold beyond coding domains. We introduce \textbf{hyperagents}, self-referential agents that integrate a task agent (which solves the target task) and a meta agent (which modifies itself and the task agent) into a single editable program. Crucially, the meta-level modification procedure is itself editable, enabling metacognitive self-modification, improving not only the task-solving behavior, but also the mechanism that generates future improvements. We instantiate this framework by extending DGM to create DGM-Hyperagents (DGM-H), eliminating the assumption of domain-specific alignment between task performance and self-modification skill to potentially support self-accelerating progress on any computable task. Across diverse domains, the DGM-H improves performance over time and outperforms baselines without self-improvement or open-ended exploration, as well as prior self-improving systems. Furthermore, the DGM-H improves the process by which it generates new agents (e.g., persistent memory, performance tracking), and these meta-level improvements transfer across domains and accumulate across runs. DGM-Hyperagents offer a glimpse of open-ended AI systems that do not merely search for better solutions, but continually improve their search for how to improve.
Here is a breakdown - https://vectree.io/c/plant-self-incompatibility-logic
But this idea of having a task agent & meta agent maybe has wings. Neat submission.
When you can nest 10+ agents deep and guarantee you will get back home without losing any data in any of the stack frames, the ability to chunk through complex problems goes up dramatically.
https://gist.github.com/ontouchstart/40db1244c0ff523f830df1c...
I've had to really shift how I think about building code bases, alot of logic can go into claude skills and sub agents. Requires essentially relearning software engineering