Hacker News Reader

Comment by danny_codes | Hacker News Reader

I suspect FOSS will converge/create a best-in-class harness within the next 6 months or so. As will most (all?) software, the FOSS stuff somehow always ends up being way better than the closed source stuff.

View on HN

In the fullness of time, LLMs will eventually become commodities and their price will be a race to the bottom. As the competitive advantage of one LLM over another shrinks - particularly as open source models expand - the value proposition will turn to the neuro/symbolic harnesses that use those models, structures that encompass agents, agent orchestration, tooling, and component plugins. Even those from frontier AI organizations will fall to open source frameworks. The last thing standing will be those who command the data centers on which these things run, and even then, that will become a commodity, with the number of players shrinking to just a few survivors. They too will find pressure: as computational power rises, more and more such systems will be run locally. - Grady Booch

Is there any resource or benchmark actually comparing how different harnesses perform with a given model?

Really just feels like endless FOMO with how fast the iteration cycle is for harness and model development.

I found this: https://artificialanalysis.ai/agents/coding-agents#harness-c...

I don't think it's very good though. As an example I can use one CC instance to delegate to several to achieve complicated/open-ended goals.

That just isn't possible with other harnesses, and it's definitely not benchmarked.

I am not sure -- Anthropic probably does benefit from the vertical integration here. They can train their models to work better with CC, which isn't really an option for third party harnesses.

With that said I think Codex/Kimi code are all behind CC as well, so maybe it's a question of effort and not the model.

Reply on news.ycombinator.com