What's the largest software project AI can complete on its own?

I was going to say that it's pretty much exactly like working with an experienced software engineer ... except it never seems to take a step back and look at the bigger picture. It will happily fix the same class of bugs over and over without realizing a bigger change is required.

The only way I've found to make it work on long term AI-code-only projects is to have it constantly refactor, with the axes of refactoring usually provided by me.

> tons of duplication that lead to whack-a-mole

The future is duplication. Code is no longer meant to be read by humans but by machines.

Please check out the code and let me know what you think. I have driven a few large refactorings that I don't think Claude would have done on its own. The two big ones were related to handling tokenization and parsing (Claude's original code had circular dependencies and would scan forward in the lever instead of using the parser to determine structure) and the second was for using subshells for command substitution instead of inline and having dual FD and buffer handling.

The current project I am developing with AI is so bloated that I've spent more time and tokens cleaning up the repo (to the tune of tens of thousands of lines removed) than actually creating it in the first place. AI is very very happy to create a mess.

I've been doing a lot of experimenting with this with a c++ compiler project[0]. I definitely experienced a lot of challenges, but I do believe it is possible to end-up in a consistent space by putting in some very careful guardrails. I covered some of the things i learned in the follow on blog posts. My more recent runs have been producing better code after continuing to improve harnesses, tests, and prompts. New blog post coming soon on v2 and v3 reruns. I'll be kicking off some v4 runs soon. Here is a teaser showing the later runs: https://storage.googleapis.com/ralph-run-viewer-zippy-960/in...

[0]: https://medium.com/@vishvananda/i-spent-2-billion-tokens-wri...

IME you can get good results if:

1. You have very good tests. The agent can write them, but you have to prompt it to do that, and some human input may be needed here.

2. You have a good, modular architecture. Again, this is something that needs to be prompted and/or developed up front. Agent work can scale if you divide and conquer, so that any given task has a well-defined scope, limiting the amount of context needed to handle it well. If you let an agent just build a big pile of code, you’ll have issues.

Once you have the above, you’re more likely to be able to just let the agent run.

This is why you hear about things like SQLite being ported to Rust (by multiple groups!) It has a comprehensive unit test suite and a good existing architecture. Such porting projects are benefitting from already having done the upfront work that I mentioned, plus they have the original source code as a reference implementation.

Because of that, most of what I do involves a lot of interaction with the agent. That’s because I’m typically working on complex existing systems, that don’t necessarily have the above properties, and agents aren’t at the point where they can handle that autonomously.

If you just say “implement this” you’re going to have a bad time after a certain point.

You really need to add yourself as a human in loop to be in the middle of design choices. That is, ask the model for a plan, what the trade offs are, should parts of the code be refactored before the next feature, etc.

Also, there’s an element of reading the code and ordering refactors. I’ve noticed that for embedded code codex loves to to do everything in a main.c and too many compiler defines. Asking it to propose a refactoring into modules helps a lot.

It’s just like writing real code, if you don’t do design up front and don’t aggressively refactor as the requirements change, it becomes a mess.

> Not to mention that if I, human overseer, don't course correct it many times a day, it goes off into Neverland.

Doesn't that happen to a human engineering team if you don't course correct it many times a month (which in code output is similar to an LLM in a day)?

> Over the last year, every Cursor/Claude/Codex project I've worked on devolves into shitty, grafted-on, duct-tape and bandaid'ed architecture

I've noticed this pattern as well when people decide to vibe code a large project.

They usually get their MVP quickly, but as soon as they want a new feature or to fix a bug, the LLM takes the shortest possible route to a solution. The problem is that after about a dozen rounds of "the honest fix" and "load-bearing code" (insert your favorite claudeism) it's a tangled mess of enterprise-grade spaghetti.

I think it's because the people full-on vibe coding just don't have the experience or time to design large scale systems and anticipate what might cause problems if they want to change something later.

IMO coding agents can be useful, but the only safe place is the manageable changes you would design up and hand off to a junior anyway (but can't because your company decided to outsource years ago).

It took about four months. Correctness was reviewed continuously rather than only after implementation. We built the system incrementally in small slices, with humans handling the architecture, and review while AI performed much of the implementation and front end is fully doe by humans.

The ingestion side includes durable asynchronous processing, recovery and replay, idempotency, strict ordering, structure aware segmentation, parent child relationships, and batched local embedding. The retrieval side includes hybrid (rrf), text and vector search, document versioning, collection and category scoping, document level ACL enforcement, parent expansion, provenance, and citation tracking.

Your estimate is fair. But, reliable WAL and retrieval layer are each substantial projects. We did not build the underlying database query engine from scratch, but the correctness work around ingestion, recovery, hierarchy, security, ranking, and provenance was still significant.

The project now has more than 3,000 automated tests, including coverage for recovery, concurrency, ordering, ACLs, versioning, and retrieval behaviour. Tests are not proof of correctness, but incremental implementation, explicit invariants, failure testing, and continuous human review are how we built confidence in it. We also had script to automate live endpoint tests with real data samples, this help largely to mitigate issues after a refactoring or enhancement. total production code is 80k LOC and rest is test coverages. One thing to say after seeing the results are, many of the RAG framework does it wrong. We got exceptional results and we also have agentic retrieval for answering complex queries. We designed it in our own way because government systems cannot afford false results, especially when the product is intended for crime investigation or compliance management in banks.

Edit: I forgot to mention that we also support PostgreSQL.

After building this system, we concluded that vector dimensionality is only a small part of retrieval accuracy. We use 384 dim embeddings and still achieve excellent results.

The tests are based on actual expected outputs, not generated only to increase coverage. For every feature, we have a phase in TASK Ledger with scope, objective, goals and outcome. And the final part will be test coverage with expectations. I cannot claim that a human has reviewed every test line by line. However, humans defined and reviewed the expected behaviour, architecture, invariants, and feature outcomes against which those tests were written. Also, when a bug is reported, we add a regression and prove the bug and then fix and test should pass.

We also use a headless API with a fully decoupled frontend, and the frontend is written entirely by developers. This helps catch additional issues during integration because every endpoint is exercised and validated through the actual userfacing workflows.

I may not be able to convince developers about the process we follow, but we got it working, and tested for concurrency, load and fanout behaviour. This is our primary requirement, as this is targeted towards large scale government customers. Regarding future tests, they will be added for new features, reported bugs, newly discovered failure modes, and regressions. This level of test coverage also helps us use AI without allowing significant drift in the codebase.

I apologize for such a big program — I didn't have enough tokens to write a small one.

- my coworkers

Thanks, your comment got this song stuck in my head. (I like my gym membership, thank you very much. I even visited the gym once several months ago.)

We wasted every second dime On diets, lawyers, shrinks, and apps, and flags And plastic surgery

- So long & thanks for all the fish, a perfect circle

I don't believe it is. If we think of AI-maintained software as automated gardening / automated agriculture, the harness is all the machinery that can plow, sow, water, monitor, harvest, etc. There still has to be a human stewarding the overall ecosystem and tending the garden. The human is 'on top' of loop rather than 'in it' but I don't think a harness by itself can be a good steward of architecture.

Product org problems also is very context-dependent. Product orgs can work significantly different, in some orgs humans desire to be deep in the details, in other orgs the product folks are happy to let engineering drive the boat beyond a certain level of complexity. So one size harness can't necessarily fit all.

I agree, Opus 5 is terrible. Worse than Opus 4.8.

Agreed. Novel would be very impressive. AI tech is struggling with that.

It is important not to wait for today's AI tech to get to AGI. It is better to exploit the useful things it can do today while waiting for it to get better tomorrow.

Cloning existing software might be useful today. It allows the model to

1) learn from its mistakes

2) mix known solutions (embed product A inside product B)

3) mine value (turn product A into a set of reusable libraries)

etc.

How would 100% non-human code possibly be safer?

I've noticed that when only AI writes the code, there are cases where it ends up being unusable by humans. For small code snippets or function-level units, AI is often much better than me. But when I look at the whole picture, AI tends to perform poorly.

Category error. Self-driving cars are still meticulously coded by humans.

I have a similar problem.

I'm not sure about architecture. When I first implemented the slot system in my language, I designed it around that implementation, and in the end, the architecture suffered. There were quite a few cases where consistency was simply unavoidable.

It's not easy to create a completely new architecture with something that doesn't exist in existing languages. Consistency doesn't come together as easily as you'd think.

In that sense, I really admire the people who first introduced something new.

When I started implementing it, I focused on one killer feature and just went for it—with the feeling that if this succeeds, my language will succeed. But once I started building around it, it turned out to be very different from other compiler architectures.

With CRUD or other well-established domains, it's easier to maintain consistency. But achieving internal consistency in a program seems to be a different kind of challenge.