waleedk

I did. Got zero response. The data would seem to indicate some tailoring is required ;-).

Here's the original not tailored post: https://news.ycombinator.com/item?id=47167393

Useful criticism -- what could I have done to help you get that message sooner?

In a sense? This human built a system for AI to build stuff then asked the AI to summarize what the AI that built the human built?

It was more of a conversation, but it was like: Hey I wrote these 6 points about what we're doing differently, please tailor them to be most useful to an HN audience.

Ha! I would rather fight 100 workhorse AIs with an Architect + Builder AIs on my side :-).

Seriously, the agents managing agents thing works so well. When I'm working, I'll sometimes have 6 builder agents fixing different bugs, and I will lose state and I rely on the architect agent who doesn't have stupid limitations like 7 +/- 2 things in working memory.

Yes, this is spec kit on steroids. In particular specs + protocol enforcement works _really_ well. The protocol enforcement is the game changer: I would find the AI just wouldn't stick to specs or plans.

Great suggestions. I will do that. Did you notice any specific issues in those?

Got it about the core flow. Appreciate it. I plan to record a video showing how to kick off a new project and another one showing how to use it in maintenance mode. Would that be helpful?

@yodon if you would like to reach out to me at hello@cluesmith.com I'd love to get your feedback once those assets are ready.

Happy to answer any questions. Here are those links as clickables:

Github: https://github.com/cluesmith/codev Tour + raw results: https://cluesmith.com/blog/a-tour-of-codevos/

Hey HN, I'm Waleed. CodevOS is the system I've been building to explore a question: what happens when you stop thinking of AI as a coding assistant and instead think of the problem of having a human-AI joint software dev team?

The 106 PRs in 14 days was one person — me — with AI agents doing the implementation. The article walks through the ideas that make this work:

- Multi-model review: Three independent AI models (Claude, Gemini, Codex) review every phase. They catch different things — Codex finds security edge cases, Claude catches runtime semantics, Gemini catches architecture problems. No single model found more than 55% of the bugs.

- An agent that helps you organize agents. You work with an Architect agent that spawns Builder agents that work simultaneously in isolated git worktrees. While one is implementing a feature, another is fixing a bug, and you're reviewing a third's PR. Your job shifts from writing code to keeping the pipeline fed.

- Natural language is the source code. Specs, plans, and reviews are version-controlled in git alongside the source code — treated with the same rigor as the code itself. The AI's instructions live in the repo, not in someone's chat history that's already been compressed. You always know why something was built and how it was designed.

- Deterministic execution. Instead of asking the AI to follow a process and hoping it does, a state machine (Porch) enforces it. Human gates, build-verify loops, mandatory review phases. The AI can't skip steps, and if it exhausts its context window, the next agent picks up from the exact checkpoint.

- Annotation over editing. Most of the work is writing and reviewing these natural language documents — specs that define what to build, plans that define how. The documents guide the agents. You're directing, not coding.

- Whole lifecycle, git at the center. From idea through specification, planning, implementation, review, PR, and merge — the entire development lifecycle is managed. Git is the backbone: worktrees for isolation, branches for workflow, PRs for integration.

It's free and open source:

npm install -g @cluesmith/codev

(and https://github.com/cluesmith/codev for the code)

The article includes a controlled comparison against unstructured Claude Code — honest about the tradeoffs (costs more, takes longer, but catches more bugs and ships with tests).

I'm genuinely looking for feedback on this. What resonates? What doesn't? What would you do differently? This is still early and I want to hear what the HN community thinks. Happy to answer questions too.

Most IDEs are built for humans, with AI bolted on as an assistant. Agent Farm is built for both: AI agents and humans share the same terminal, same commands, same workflow. We shipped 60 features using it to build itself.

GitHub: https://github.com/cluesmith/codev

Quick start: `npm i -g @cluesmith/codev && af start`

How it works:

The Architect window is your control center. You create specs (what to build) and plans (how to build it). Then with `af spawn --project 22`, a builder agent spins up and implements the spec. When it's done, it opens a PR – already reviewed by a panel of AI agents (Gemini, Claude, GPT). It notifies you, and you merge.

Meanwhile, you're not blocked. You can spawn more builders, design the next feature, or review what's already done.

How it's built:

Agent Farm runs in your browser, built on common infrastructure:

- tmux – each agent gets its own terminal session

- ttyd – exposes terminals to the browser (no Electron)

- git worktrees – each builder gets an isolated worktree on its own branch

~20K lines of TypeScript + HTML. VSCode is 2.7 million.

The workflow:

Codev structures the work: `codev init` sets up specs/, plans/, reviews/ folders. You write specs first, then plans. Once approved, spawn one builder per spec. Three specs? Three builders, parallel, isolated. Each opens a PR when done.

Opinionated design:

- Simple extensible implementation

- No chat sidebar – agents get full terminal sessions

- Specs and plans are markdown files

- Multi-agent review before human review

Live Demo: youtube.com/watch?v=0OEhdk7-plE

Would love feedback on it. Happy to answer questions.

Thanks! Appreciate the link!

Author here. Got tired of hearing so much about AGI, when the problems with AI today are more pressing.

We’re still learning how to integrate chat interfaces in existing web apps. A lot of big companies are getting this badly wrong (including Google and Meta). I posit 4 principles of successful chat integrations:

- Must be able to operate on the core primitives of the app

- Must take advantage of the powerful characteristics of natural language

- Allow the user to fluidly mix and match between the traditional UI interface and the chat interface

- Must maintain context across the user’s entire experience.

I discuss each of these with concrete examples of existing products, and also with a simple idea of a chat interface-enabled todo list manager. I also discuss some litmus tests. But if you want the short take-away: imitate Notion, and avoid the mistakes that Google Docs has made.

I think of context as something dynamic that you build as you need it. Part of getting something like Claude Code to the point where it can produce great results is selecting different parts to emphasize.

Intent is important, but so are goals, and I see the context as something that you collaboratively build with you filling in the gaps in the context the agent doesn't see.

It's almost like you repetitively ask the question: based on how the agent is behaving, what is the context I should add to help it perform better? If it's screwing up how to use an API, then I should ask it to read the docs and create a guide for itself or provide it with an llms.txt.

Author here. Happy to answer any questions you have.

I know the HackerNews crowd is anti-AI in the workforce overall, but I wanted to share a different approach which is optimizing for collaboration between humans and AI. This maps into certain practical things you can do, including things like how you structure your repo.

[Original poster here] I agree in the wrong hands it can be dangerous. If you really think knowing the package visibility of a static variable in Java is the critical thing to succeed in your role, then you have bigger problems.

[Author] The problem is: how could you objectively evaluate the factuality in any other way? Summary ranking is an established approach in the literature.

[Author] This is what I meant. That even if it doesn't work at a single layer, you could build a "verifier" stage whose approach is similar to this verification process and it can make the decision.

[Author] As an end user, you pay the published price of $1 per million tokens for alternatives (such as Anyscale Endpoints -- https://app.endpoints.anyscale.com/landing, but there are others -- Replikate, Fireworks etc). This isn't just the compute price -- there's some margin in there as well.

[Author] I acknowledge this and the article does as well. What I would say is that -- at the very least -- you could apply factual verification as a post process (so stage 1 is create the summary, stage 2 is validate the summary is factually accurate).

[Author] OpenAI uses Ray, the open source version of Anyscale Platform, for training their models (video evidence: https://www.youtube.com/watch?v=CqiL5QQnN64).

Ray (https://github.com/ray-project/ray) is available for anyone to use. You can also use Aviary (https://github.com/ray-project/aviary) to serve any of those models yourself.

[Author] I see it differently. I see that Anyscale built some really great infrastructure for scaling AI that could be applied to many domains. What we've seen is the market shift to LLMs, and we've been able to adapt our tech to new domains.

Our prices are competitive (starting at 25c per million tokens) because of the tech we've built that maximizes GPU utilization. That's what we do better than anyone else.

I still think GPT-4 is best LLM out there. It's just very expensive and you don't need all that horsepower you can save bucketloads of money.

[Author] Can you be specific about which part of the research is dishonest?

We've shared the code openly, you can reproduce it yourself if you want.

[Author] Can you cite some examples of this so we're not talking in the void?

I've actually tested Llama 2 for summarization but haven't blogged about that yet, and across multiple domains, Llama 2 is pretty good.

I do see some differences -- Llama 2 doesn't follow instructions as cleanly, and is more verbose.

It's not a plugin replacement -- in the article itself I point it out. GPT 3.5 largely followed instructions to return A or B. Llama 2 didn't so I had to use another LLM to post-process.

I'm not saying you should always use Llama 2 or ChatGPT. It's that for some use cases, you can save a lot of money by using an open source alternative.

[Author] Can you share more details? How does it fail? What type of domain? Perhaps there is some tweaking to prompts that's required? It would help to understand what you're seeing.

[Author] Can you elaborate on what you mean here? I'd like to understand what you mean by "blind benchmark" and "not public"?

[Contributor to Aviary] Not many are building their own LLMs (though there are some like Bloomberg). But quite a few are experimenting with fine tuning.

One of the great things with LLMs is that they can (and are) fine-tuned with small amounts of data. In our previous work, we showed it cost as little as $7 to fine tune a model with 40,000 lines of text.

[Author] Good luck trying to use clusters of Lambda machines. Lambda labs are cheap for a reason: their API is not very featureful (we looked at them and we saw they didn't even support machine tagging). If you're looking for a box or two, lambda labs is fine. If you're looking for 1,000, not so much.

Plus they don't actually have any actually A100s available at the moment (2022-05-17).

CoreWeave is a nice middle ground. You can at least get the A100 machines into a k8s cluster.

[Author] Mosaic must be getting some kind of sweetheart deals on A100 80GB and A100 40GB. The prices they are quoting are not what say the AWS on-demand prices are. They quote $2 per GPU for A100 40GB and $2.50 for A100 80GB. That's literally half the AWS on-demand rate for A100s here: https://aws.amazon.com/ec2/instance-types/p4/

And these are impossible to get. We tried to get some for Anyscale, and we were told there were no on-demand available and lead time for reserved (ouchie on the price! You're talking a quarter of a million dollars a year for one machine at list) was in weeks.

Once you take the model size and hefty sweetheart deals into account, you're within 10%. Mosaic does have some nice whitebox optimizations, but nothing that radically changes the equation.

[Author] TL;DR OS LLM models are coming.

Dolly's not that great -- I've hit lots of issues using it to be honest .

MosaicML has a nice commercially usable model here: https://www.mosaicml.com/blog/mpt-7b

I think they're one of the leading ones (bias: they're kinda competitors to my employer Anyscale, but you gotta say something's good when it is).

Red Pajama are leading an effort to build a fully open source model similar to LLaMa. https://www.together.xyz/blog/redpajama

[Author] You approximate the weights using fewer bits. You also switch to ints instead of floats and then do some fancy stuff when multiplying to make it all work together.

More detail than you probably wanted: https://huggingface.co/blog/hf-bitsandbytes-integration

[Author] Fair point -- I clarified the language and gave a concrete example. Hope that helps!