Hacker News Reader

Comment by mccoyb | Hacker News Reader

I don’t think it’s possible to build a software factory unless you truly do not care about what you’re putting out.

Agent swarms, self learning, Ralph loops, execution DAGs, spending hours trying to convey my preferences into skills, yada yada — the reality is that no matter how long I spend trying to close the amortization gap between my expertise and (insert autonomous agent loop) … I always come back to see shit that I have opinions about and need to polish.

If you’re keeping on top of your craft, I just don’t think general purpose pretraining / RL is going to give you something which perfectly copies you. It can get closer and closer! But I’ve never seen agent work where I didn’t immediately start thinking of refinements …

I guess many things are okay if you’re okay with averages

View on HN

The fascination with "software factories" is strange. We don't have "bridge factories" or "skyscraper factories." We engineer those for the specific requirements from each individual project.

I think it's rooted in the idea that AI is going to "solve" software engineering when it's really only another tool that magnifies leverage. Same thing with traditional engineering and their computer-assisted tools. It doesn't "solve" engineering, but it does automate some of the repetitive things that aren't the biggest value-add.

Software isn't a manufactured product and that's where the disconnect is. You can't build software factories unless you don't care about the quality. For some companies, that might be fine, but I imagine that's a much smaller subset of the market than folks think.

> I don’t think it’s possible to build a software factory unless you truly do not care about what you’re putting out

In the article they talk of a swarm of agents autonomously speccing out the software and I can't help but wonder -- at that point, what are you creating, and for who??

> I always come back to see shit that I have opinions about and need to polish

Definitely but can't the factory analogy still apply here?

We're now workers on a factory assembly line.

The assembly line still needs people to design and build machines, press buttons to operate machines, verify widgets are correct for their stage of assembly, and verify the final product is correct and ready to leave the assembly line.

Importantly it needs people that spot defects and send things back, and press the "big red button" if something is grossly wrong with the whole process.

The goal is to make this as efficient as possible working with our agents.

I don't think the agentic coding tools are anywhere close to making this awesome yet, but at the same time the best practices of "dev ops" need to be updated to involve agents.

Agree with you, overall -- and it's worthwhile to learn how to setup factories at various scales of problems, and one will always be in a situation where you need to decide whether you care about a particular part of the process, or can be okay with averages.

I'm only saying that that I don't think one can automate the whole thing. After all, agents don't actually possess ... agency -- they require us to breath live into them. So at some part of the stack, there has to be a human, with an opinion, thinking about what they want.

Also: I don't aspire to be a worker, I aspire to seize the means of production.

> Also: I don't aspire to be a worker, I aspire to seize the means of production.

You are not alone. Let's do it!

Despite the marketing, the "I" in AI doesn't (yet) stand for "intelligence". You're best off finding applications where you can easily write a rigorous, well-defined, easily evaluated acceptance criterion and then let the model, harness, etc go wild. This of course is also the case with "AI coding" technology that came before--genetic programming, NEAT, etc. It's not intelligent, and therefore limited in fundamental ways. You have to steer it to make it work. This is very fundamentally different from how factory workers operate.

Therefore I can't see how the "software factories" concept is actually feasible without a technological breakthrough. We may be no closer to it now than we were 25yr ago.

I agree. I speak from experience because I actually went down this path for about a month. It is tempting. However, what I realized after some reflection is that there are just so many decisions that need to be made along the course of developing a product that it's never going to be possible to just one-shot it. There's a continuous feedback loop that is needed and agents aren't going to be the ones filling that role.

> I always come back to see shit that I have opinions about and need to polish

But isn't this the case with all code?

Same experience. I've spent hours tweaking prompts, harnesses, skills and what not to have a simple (but non trivial) project completely managed by an agent.

The key constraint was keeping the quality bar high.

If I lower the quality bar and _only_ care about a working solution, this works great. And I dare not look at the implementation details if this is the outcome I care about.

But if I want a working solution _and_ have it done the right way, then I need to be in the loop at key points. Newer models, unfortunately, have made this problem worse. I will inevitably come back to some form of bad decisions, deviations or just slop. Left unattended, it snowballs into a pile of mess. At this point, I decide if I spend time fixing it or switch my mental model to not care about the quality.

The problem is worse when there's a team involved here. Fragmented tooling , prompts, harness, model causes more divergence. Especially in quality. Senior engineers become bottleneck as the quality enforcers.

I do like the revert rate metric that Posthog tracks. This can help incrementally build the tooling to a point it can be trusted and scales.

Reply on news.ycombinator.com