selridge

It is WILD how much better things got when I ripped out all the "best practices" from https://agents.md/

>"as AI becomes more agentic, we are entering a new era where software can, in a very real sense, become self-improving."

>"This creates a continuous feedback loop. When an AI agent implements a new feature, its final task isn't just to "commit the code." Instead, as part of the Continuous Alignment process, the agent's final step is to reflect on what changed and update the project's knowledge base accordingly."

>"... the type of self-improvement we’re talking about is far more pragmatic and much less dangerous."

>"Self-improving software isn't about creating a digital god; it's about building a more resilient, maintainable, and understandable system. By closing the loop between code and documentation, we set the stage for even more complex collaborations."

It's only like every other sentence.

I'm not really trolling. I'm trying to push people to consider that the world is already in a state where "I used AI" is neither binary nor dispositive. I think we're used to a 2023 to mid-2025 framing where outside of some narrow, highly structured cases, the code is garbage.

If that's still true as a binary now, it won't be for long. As the robot likes to say, some of these changes are "in flight".

That paper is hard to evaluate because their modal example of a “context file” is a bad practice that arose from early attempts at text-based agent guidance before we recognized that contextual instruction was the aim, not just “context.”

With a “context file” you’re almost guaranteed to add bloat without useful behavior change because it’s just a pre-set list of things that could be good to know about.

So the results of the study don’t generalize to every text file used for instruction or even most.

But they're not your agents.

Thanks but no thanks.

What doesn't that explain tho?

What behavior would you need to see for that explanation to no longer hold? Because it seems like it explains too much.

It's an octopus, perhaps: https://aclanthology.org/2020.acl-main.463.pdf

There's also this blog post: https://julianmichael.org/blog/2020/07/23/to-dissect-an-octo... (which IMO is better to read than the paper)

Yes. I’m saying that “it’s just in the training data” is a cognitive containment of these models which is incomplete. You can insist that’s what’s happening, but you’ll be left unable to explain what’s going on beyond truisms.

Ok cool cool. Instead of pretending you need to teach me, you could engage with what I'm saying or even the OP!

"I don't know how you get here from "predict the next word"" is not really so much a statement of ignorance where someone needs you to step in but a reflection that perhaps the tech is not so easily explained as that. No magic needs to be present for that to be the case.

>You're implicitly assuming that what you asked the LLM to do is unrepresented in the training data.

This is just as stuck in a moment in time as "they only do next word prediction" What does this even mean anymore? Are we supposed to believe that a review of this paper that wasn't written when that model (It's putatively not an "LLM", but IDK enough about it to be pushy there) was trained? Does that even make sense? We're not in the regime of regurgitating training data (if we really ever were). We need to let go of these frames which were barely true when they took hold. Some new shit is afoot.

Do you see a problem with a non-programmer writing code?

I hear it.

"There is nothing independent about it"

Ok. We'll see how that shakes out in a year or so. Most folks in the west have never lived in a country where a dictator controls the money supply. It's REAL easy to shit talk the US or even european central banks are when you have no idea what the alternative is.

And what does this have to do with the price of tea in China?

Pack it in everyone, the “OK for prod use” guy has spoken.

This is satire and therefore must be judged on brevity and comedy. It fucked both up.

We need to RETVRN to 120 char.

This article is far off the mark. The improvement is not in the user-side. You can write docs or have the robot write docs; it will improve performance on your repo, but not “improve” the agent.

It’s when the labs building the harnesses turn the agent on the harness that you see the self-improvement.

You can improve your project and your context. If you don’t own the agent harness you’re not improving the agent.

LMFAO what is this

"show the prompts"

What would the prompt for this look like?

Great write-up. Hilarious situation where no one (except unwieldiness) is the villain.

Yeah, I got that. It was subtle, but I managed.

the closest thing we have to explaining the issue of focusing on something is "imagine a CPU from the 1990s"?

C'mon.

Yeah sounds like he was really scrimping.

Take that up with the CA voters circa 1979.

I hope he sees this, bro.

Better dead in California than alive in Florida, to paraphrase.

0 to “we are even talking about this” is an astonishing leap. Acting like this stuff has been standing still is an active choice.

No questions but I appreciate the write-up! Thank you for sharing.

Alignment is impossible here. “Nearly certain” odds for success for a sports team might be 20:1, but that’s a little worse (not much!) than for a launch vehicle and not at all good for a web server. No one would say “it is nearly certain that I’ll serve a web request” based on two 9’s, but they would say “it is nearly certain the team will win today” given the same odds. That’s just between humans.